[jira] [Commented] (HIVE-19306) Arrow batch serializer
[ https://issues.apache.org/jira/browse/HIVE-19306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16494249#comment-16494249 ] Vineet Garg commented on HIVE-19306: Pushed this to branch-3. Test run in HIVE-19305 > Arrow batch serializer > -- > > Key: HIVE-19306 > URL: https://issues.apache.org/jira/browse/HIVE-19306 > Project: Hive > Issue Type: Task > Components: Serializers/Deserializers >Reporter: Eric Wohlstadter >Assignee: Teddy Choi >Priority: Major > Fix For: 3.1.0, 4.0.0 > > Attachments: HIVE-19306.2.patch, HIVE-19306.3.patch, > HIVE-19306.4.patch > > > Leverage the ThriftJDBCBinarySerDe code path that already exists in > SemanticAnalyzer/FileSinkOperator to create a serializer that batches rows > into Arrow vector batches. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19306) Arrow batch serializer
[ https://issues.apache.org/jira/browse/HIVE-19306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471444#comment-16471444 ] Teddy Choi commented on HIVE-19306: --- [~mmccline], thanks for committing! > Arrow batch serializer > -- > > Key: HIVE-19306 > URL: https://issues.apache.org/jira/browse/HIVE-19306 > Project: Hive > Issue Type: Task > Components: Serializers/Deserializers >Reporter: Eric Wohlstadter >Assignee: Teddy Choi >Priority: Major > Fix For: 3.1.0 > > Attachments: HIVE-19306.2.patch, HIVE-19306.3.patch, > HIVE-19306.4.patch > > > Leverage the ThriftJDBCBinarySerDe code path that already exists in > SemanticAnalyzer/FileSinkOperator to create a serializer that batches rows > into Arrow vector batches. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19306) Arrow batch serializer
[ https://issues.apache.org/jira/browse/HIVE-19306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471166#comment-16471166 ] Matt McCline commented on HIVE-19306: - Committed to master. > Arrow batch serializer > -- > > Key: HIVE-19306 > URL: https://issues.apache.org/jira/browse/HIVE-19306 > Project: Hive > Issue Type: Task > Components: Serializers/Deserializers >Reporter: Eric Wohlstadter >Assignee: Teddy Choi >Priority: Major > Fix For: 3.1.0 > > Attachments: HIVE-19306.2.patch, HIVE-19306.3.patch, > HIVE-19306.4.patch > > > Leverage the ThriftJDBCBinarySerDe code path that already exists in > SemanticAnalyzer/FileSinkOperator to create a serializer that batches rows > into Arrow vector batches. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19306) Arrow batch serializer
[ https://issues.apache.org/jira/browse/HIVE-19306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471165#comment-16471165 ] Matt McCline commented on HIVE-19306: - +1 LGTM > Arrow batch serializer > -- > > Key: HIVE-19306 > URL: https://issues.apache.org/jira/browse/HIVE-19306 > Project: Hive > Issue Type: Task > Components: Serializers/Deserializers >Reporter: Eric Wohlstadter >Assignee: Teddy Choi >Priority: Major > Attachments: HIVE-19306.2.patch, HIVE-19306.3.patch, > HIVE-19306.4.patch > > > Leverage the ThriftJDBCBinarySerDe code path that already exists in > SemanticAnalyzer/FileSinkOperator to create a serializer that batches rows > into Arrow vector batches. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19306) Arrow batch serializer
[
https://issues.apache.org/jira/browse/HIVE-19306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16470122#comment-16470122
]
Hive QA commented on HIVE-19306:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12922558/HIVE-19306.4.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 43 failed/errored test(s), 13574 tests
executed
*Failed tests:*
{noformat}
TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed
out) (batchId=247)
TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed
out) (batchId=247)
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=95)
[jira] [Commented] (HIVE-19306) Arrow batch serializer
[
https://issues.apache.org/jira/browse/HIVE-19306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16470051#comment-16470051
]
Hive QA commented on HIVE-19306:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
42s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
29s{color} | {color:blue} common in master has 62 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
36s{color} | {color:blue} serde in master has 190 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
45s{color} | {color:blue} ql in master has 2321 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
19s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
38s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
44s{color} | {color:red} ql: The patch generated 588 new + 0 unchanged - 0
fixed = 588 total (was 0) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
29s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
55s{color} | {color:red} ql generated 1 new + 99 unchanged - 1 fixed = 100
total (was 100) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
13s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 29m 41s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile
xml |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10797/dev-support/hive-personality.sh
|
| git revision | master / 1cd5274 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10797/yetus/diff-checkstyle-ql.txt
|
| javadoc |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10797/yetus/diff-javadoc-javadoc-ql.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10797/yetus/patch-asflicense-problems.txt
|
| modules | C: common serde ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10797/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Arrow batch serializer
> --
>
> Key: HIVE-19306
> URL: https://issues.apache.org/jira/browse/HIVE-19306
> Project: Hive
> Issue Type: Task
> Components: Serializers/Deserializers
>Reporter: Eric Wohlstadter
>Assignee: Teddy Choi
>Priority: Major
> Attachments: HIVE-19306.2.patch, HIVE-19306.3.patch,
> HIVE-19306.4.patch
>
>
> Leverage the ThriftJDBCBinarySerDe code path that already exists in
>
[jira] [Commented] (HIVE-19306) Arrow batch serializer
[ https://issues.apache.org/jira/browse/HIVE-19306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16468155#comment-16468155 ] Teddy Choi commented on HIVE-19306: --- I fixed the static assignment that FindBugs warned. > Arrow batch serializer > -- > > Key: HIVE-19306 > URL: https://issues.apache.org/jira/browse/HIVE-19306 > Project: Hive > Issue Type: Task > Components: Serializers/Deserializers >Reporter: Eric Wohlstadter >Assignee: Teddy Choi >Priority: Major > Attachments: HIVE-19306.2.patch, HIVE-19306.3.patch, > HIVE-19306.4.patch > > > Leverage the ThriftJDBCBinarySerDe code path that already exists in > SemanticAnalyzer/FileSinkOperator to create a serializer that batches rows > into Arrow vector batches. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19306) Arrow batch serializer
[
https://issues.apache.org/jira/browse/HIVE-19306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16467188#comment-16467188
]
Hive QA commented on HIVE-19306:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12922288/HIVE-19306.3.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 222 failed/errored test(s), 13569 tests
executed
*Failed tests:*
{noformat}
TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed
out) (batchId=247)
TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed
out) (batchId=247)
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=95)
[jira] [Commented] (HIVE-19306) Arrow batch serializer
[
https://issues.apache.org/jira/browse/HIVE-19306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16467135#comment-16467135
]
Hive QA commented on HIVE-19306:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
34s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
51s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
37s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
7s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
32s{color} | {color:blue} common in master has 62 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
41s{color} | {color:blue} serde in master has 190 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
12s{color} | {color:blue} ql in master has 2323 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
24s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
36s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
43s{color} | {color:red} ql: The patch generated 588 new + 0 unchanged - 0
fixed = 588 total (was 0) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
8s{color} | {color:red} ql generated 1 new + 2323 unchanged - 0 fixed = 2324
total (was 2323) {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
58s{color} | {color:red} ql generated 1 new + 99 unchanged - 1 fixed = 100
total (was 100) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
16s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 31m 2s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Write to static field
org.apache.hadoop.hive.ql.io.arrow.ArrowColumnarBatchSerDe.rootAllocator from
instance method
org.apache.hadoop.hive.ql.io.arrow.ArrowColumnarBatchSerDe.initialize(Configuration,
Properties) At ArrowColumnarBatchSerDe.java:from instance method
org.apache.hadoop.hive.ql.io.arrow.ArrowColumnarBatchSerDe.initialize(Configuration,
Properties) At ArrowColumnarBatchSerDe.java:[line 164] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile
xml |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10760/dev-support/hive-personality.sh
|
| git revision | master / b0eb415 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10760/yetus/diff-checkstyle-ql.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10760/yetus/new-findbugs-ql.html
|
| javadoc |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10760/yetus/diff-javadoc-javadoc-ql.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10760/yetus/patch-asflicense-problems.txt
|
| modules | C: common serde ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10760/yetus.txt |
| Powered by | Apache Yetus
[jira] [Commented] (HIVE-19306) Arrow batch serializer
[ https://issues.apache.org/jira/browse/HIVE-19306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16466188#comment-16466188 ] Eric Wohlstadter commented on HIVE-19306: - +1 lgtm > Arrow batch serializer > -- > > Key: HIVE-19306 > URL: https://issues.apache.org/jira/browse/HIVE-19306 > Project: Hive > Issue Type: Task > Components: Serializers/Deserializers >Reporter: Eric Wohlstadter >Assignee: Teddy Choi >Priority: Major > Attachments: HIVE-19306.2.patch > > > Leverage the ThriftJDBCBinarySerDe code path that already exists in > SemanticAnalyzer/FileSinkOperator to create a serializer that batches rows > into Arrow vector batches. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19306) Arrow batch serializer
[ https://issues.apache.org/jira/browse/HIVE-19306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16463200#comment-16463200 ] Teddy Choi commented on HIVE-19306: --- [~mmccline], [~ewohlstadter]. [^HIVE-19306.3.patch] fixed null handling. ArrowColumnarBatchSerDe#writeNull is the strangest part. Because Apache Arrow's UnionListWriter should implement AbstractFieldWriter#writeNull properly and FieldWriter should have a super method of AbstractFieldWriter#writeNull to expose it. I used reflection and concrete class check to handle it. I'll fix FieldWriter interface in Apache Arrow. > Arrow batch serializer > -- > > Key: HIVE-19306 > URL: https://issues.apache.org/jira/browse/HIVE-19306 > Project: Hive > Issue Type: Task > Components: Serializers/Deserializers >Reporter: Eric Wohlstadter >Assignee: Teddy Choi >Priority: Major > Attachments: HIVE-19306.2.patch > > > Leverage the ThriftJDBCBinarySerDe code path that already exists in > SemanticAnalyzer/FileSinkOperator to create a serializer that batches rows > into Arrow vector batches. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19306) Arrow batch serializer
[ https://issues.apache.org/jira/browse/HIVE-19306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461599#comment-16461599 ] Eric Wohlstadter commented on HIVE-19306: - [~teddy.choi] I don't think it is necessary or useful to use FieldWriter for non-complex types. Should we change it so that non-complex types use "set" and "setNull" directly on the vector? I believe this would remove some of the indirection which is making the code harder to debug. > Arrow batch serializer > -- > > Key: HIVE-19306 > URL: https://issues.apache.org/jira/browse/HIVE-19306 > Project: Hive > Issue Type: Task > Components: Serializers/Deserializers >Reporter: Eric Wohlstadter >Assignee: Teddy Choi >Priority: Major > Attachments: HIVE-19306.2.patch > > > Leverage the ThriftJDBCBinarySerDe code path that already exists in > SemanticAnalyzer/FileSinkOperator to create a serializer that batches rows > into Arrow vector batches. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19306) Arrow batch serializer
[ https://issues.apache.org/jira/browse/HIVE-19306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461597#comment-16461597 ] Matt McCline commented on HIVE-19306: - [~teddy.choi] The part that I see NULL issues (so far) is in the ArrowColumnarBatchSerDe.write handling of selectedInUse. However, my impression is you are only writing the batch you fill from row-mode and that doesn't have selected rows. > Arrow batch serializer > -- > > Key: HIVE-19306 > URL: https://issues.apache.org/jira/browse/HIVE-19306 > Project: Hive > Issue Type: Task > Components: Serializers/Deserializers >Reporter: Eric Wohlstadter >Assignee: Teddy Choi >Priority: Major > Attachments: HIVE-19306.2.patch > > > Leverage the ThriftJDBCBinarySerDe code path that already exists in > SemanticAnalyzer/FileSinkOperator to create a serializer that batches rows > into Arrow vector batches. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19306) Arrow batch serializer
[
https://issues.apache.org/jira/browse/HIVE-19306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461449#comment-16461449
]
Eric Wohlstadter commented on HIVE-19306:
-
[~mmccline]
Can you help to review? Especially usage of
{{org.apache.hadoop.hive.ql.exec.vector}}
> Arrow batch serializer
> --
>
> Key: HIVE-19306
> URL: https://issues.apache.org/jira/browse/HIVE-19306
> Project: Hive
> Issue Type: Task
> Components: Serializers/Deserializers
>Reporter: Eric Wohlstadter
>Assignee: Teddy Choi
>Priority: Major
> Attachments: HIVE-19306.2.patch
>
>
> Leverage the ThriftJDBCBinarySerDe code path that already exists in
> SemanticAnalyzer/FileSinkOperator to create a serializer that batches rows
> into Arrow vector batches.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19306) Arrow batch serializer
[ https://issues.apache.org/jira/browse/HIVE-19306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16458965#comment-16458965 ] Eric Wohlstadter commented on HIVE-19306: - [~teddy.choi] Can you put this on Review Board? > Arrow batch serializer > -- > > Key: HIVE-19306 > URL: https://issues.apache.org/jira/browse/HIVE-19306 > Project: Hive > Issue Type: Task > Components: Serializers/Deserializers >Reporter: Eric Wohlstadter >Assignee: Teddy Choi >Priority: Major > Attachments: HIVE-19306.2.patch > > > Leverage the ThriftJDBCBinarySerDe code path that already exists in > SemanticAnalyzer/FileSinkOperator to create a serializer that batches rows > into Arrow vector batches. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19306) Arrow batch serializer
[
https://issues.apache.org/jira/browse/HIVE-19306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16455747#comment-16455747
]
Hive QA commented on HIVE-19306:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12920830/HIVE-19306.2.patch
{color:green}SUCCESS:{color} +1 due to 6 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 43 failed/errored test(s), 14320 tests
executed
*Failed tests:*
{noformat}
TestMinimrCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=93)
[infer_bucket_sort_num_buckets.q,infer_bucket_sort_reducers_power_two.q,parallel_orderby.q,bucket_num_reducers_acid.q,infer_bucket_sort_map_operators.q,infer_bucket_sort_merge.q,root_dir_external_table.q,infer_bucket_sort_dyn_part.q,udf_using.q,bucket_num_reducers_acid2.q]
TestNonCatCallsWithCatalog - did not produce a TEST-*.xml file (likely timed
out) (batchId=217)
TestTxnExIm - did not produce a TEST-*.xml file (likely timed out) (batchId=286)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_nullscan]
(batchId=68)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_table_stats]
(batchId=54)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_4]
(batchId=13)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_0]
(batchId=17)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[row__id] (batchId=80)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[acid_bucket_pruning]
(batchId=150)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[acid_vectorization_original]
(batchId=173)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[default_constraint]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dynpart_sort_optimization_acid]
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[enforce_constraint_notnull]
(batchId=158)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[materialized_view_create_rewrite_4]
(batchId=157)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[materialized_view_create_rewrite_5]
(batchId=154)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_stats]
(batchId=168)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_text_vec_part]
(batchId=168)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[acid_vectorization_original_tez]
(batchId=106)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_5]
(batchId=105)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[cluster_tasklog_retrieval]
(batchId=98)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[mapreduce_stack_trace]
(batchId=98)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[mapreduce_stack_trace_turnoff]
(batchId=98)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[minimr_broken_pipe]
(batchId=98)
org.apache.hadoop.hive.ql.TestAcidOnTez.testAcidInsertWithRemoveUnion
(batchId=228)
org.apache.hadoop.hive.ql.TestAcidOnTez.testCtasTezUnion (batchId=228)
org.apache.hadoop.hive.ql.TestAcidOnTez.testNonStandardConversion01
(batchId=228)
org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1 (batchId=232)
org.apache.hadoop.hive.ql.parse.TestCopyUtils.testPrivilegedDistCpWithSameUserAsCurrentDoesNotTryToImpersonate
(batchId=231)
org.apache.hadoop.hive.ql.parse.TestReplicationOnHDFSEncryptedZones.targetAndSourceHaveDifferentEncryptionZoneKeys
(batchId=231)
org.apache.hive.beeline.TestBeeLineWithArgs.testQueryProgressParallel
(batchId=235)
org.apache.hive.jdbc.TestSSL.testSSLFetchHttp (batchId=239)
org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testTokenAuth (batchId=254)
org.apache.hive.minikdc.TestJdbcWithDBTokenStoreNoDoAs.testCancelRenewTokenFlow
(batchId=254)
org.apache.hive.minikdc.TestJdbcWithDBTokenStoreNoDoAs.testConnection
(batchId=254)
org.apache.hive.minikdc.TestJdbcWithDBTokenStoreNoDoAs.testIsValid (batchId=254)
org.apache.hive.minikdc.TestJdbcWithDBTokenStoreNoDoAs.testIsValidNeg
(batchId=254)
org.apache.hive.minikdc.TestJdbcWithDBTokenStoreNoDoAs.testNegativeProxyAuth
(batchId=254)
org.apache.hive.minikdc.TestJdbcWithDBTokenStoreNoDoAs.testNegativeTokenAuth
(batchId=254)
org.apache.hive.minikdc.TestJdbcWithDBTokenStoreNoDoAs.testProxyAuth
(batchId=254)
org.apache.hive.minikdc.TestJdbcWithDBTokenStoreNoDoAs.testRenewDelegationToken
(batchId=254)
org.apache.hive.minikdc.TestJdbcWithDBTokenStoreNoDoAs.testTokenAuth
(batchId=254)
{noformat}
Test results:
[jira] [Commented] (HIVE-19306) Arrow batch serializer
[
https://issues.apache.org/jira/browse/HIVE-19306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16455736#comment-16455736
]
Hive QA commented on HIVE-19306:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
44s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
39s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
10s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 8m
8s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m
40s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 2m
1s{color} | {color:red} root: The patch generated 655 new + 427 unchanged - 0
fixed = 1082 total (was 427) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
47s{color} | {color:red} ql: The patch generated 655 new + 0 unchanged - 0
fixed = 655 total (was 0) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
3s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 6m
15s{color} | {color:red} root generated 1 new + 333 unchanged - 1 fixed = 334
total (was 334) {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 1m
2s{color} | {color:red} ql generated 1 new + 99 unchanged - 1 fixed = 100 total
(was 100) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
14s{color} | {color:red} The patch generated 2 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 56m 20s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile
xml |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10518/dev-support/hive-personality.sh
|
| git revision | master / 0dec595 |
| Default Java | 1.8.0_111 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10518/yetus/diff-checkstyle-root.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10518/yetus/diff-checkstyle-ql.txt
|
| javadoc |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10518/yetus/diff-javadoc-javadoc-root.txt
|
| javadoc |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10518/yetus/diff-javadoc-javadoc-ql.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10518/yetus/patch-asflicense-problems.txt
|
| modules | C: common serde . ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10518/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Arrow batch serializer
> --
>
> Key: HIVE-19306
> URL: https://issues.apache.org/jira/browse/HIVE-19306
> Project: Hive
> Issue Type: Task
> Components: Serializers/Deserializers
>Reporter: Eric Wohlstadter
>Assignee: Teddy Choi
>Priority: Major
> Attachments: HIVE-19306.2.patch
>
>
> Leverage the
