[jira] [Commented] (HIVE-19307) Support ArrowOutputStream in LlapOutputFormatService
[ https://issues.apache.org/jira/browse/HIVE-19307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16494259#comment-16494259 ] Vineet Garg commented on HIVE-19307: Pushed this to branch-3. Test run in HIVE-19305 > Support ArrowOutputStream in LlapOutputFormatService > > > Key: HIVE-19307 > URL: https://issues.apache.org/jira/browse/HIVE-19307 > Project: Hive > Issue Type: New Feature > Components: llap >Reporter: Eric Wohlstadter >Assignee: Eric Wohlstadter >Priority: Major > Fix For: 3.1.0, 4.0.0 > > Attachments: HIVE-19307.3.patch, HIVE-19307.4.patch, > HIVE-19307.5.patch, HIVE-19307.6.patch, HIVE-19307.7.patch, HIVE-19307.8.patch > > > Support pushing arrow batches through > org.apache.arrow.vector.ipc.ArrowOutputStream in LllapOutputFormatService. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19307) Support ArrowOutputStream in LlapOutputFormatService
[
https://issues.apache.org/jira/browse/HIVE-19307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16475497#comment-16475497
]
Hive QA commented on HIVE-19307:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12923322/HIVE-19307.8.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 14405 tests
executed
*Failed tests:*
{noformat}
TestNonCatCallsWithCatalog - did not produce a TEST-*.xml file (likely timed
out) (batchId=217)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[mergejoin]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_vector_dynpart_hashjoin_1]
(batchId=172)
org.apache.hive.jdbc.TestTriggersMoveWorkloadManager.testTriggerMoveAndKill
(batchId=240)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/10956/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/10956/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-10956/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12923322 - PreCommit-HIVE-Build
> Support ArrowOutputStream in LlapOutputFormatService
>
>
> Key: HIVE-19307
> URL: https://issues.apache.org/jira/browse/HIVE-19307
> Project: Hive
> Issue Type: Task
> Components: llap
>Reporter: Eric Wohlstadter
>Assignee: Eric Wohlstadter
>Priority: Major
> Attachments: HIVE-19307.3.patch, HIVE-19307.4.patch,
> HIVE-19307.5.patch, HIVE-19307.6.patch, HIVE-19307.7.patch, HIVE-19307.8.patch
>
>
> Support pushing arrow batches through
> org.apache.arrow.vector.ipc.ArrowOutputStream in LllapOutputFormatService.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19307) Support ArrowOutputStream in LlapOutputFormatService
[
https://issues.apache.org/jira/browse/HIVE-19307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16475471#comment-16475471
]
Hive QA commented on HIVE-19307:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
38s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
57s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
31s{color} | {color:blue} common in master has 62 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
35s{color} | {color:blue} ql in master has 2320 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} The patch common passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s{color} | {color:green} ql: The patch generated 0 new + 565 unchanged - 14
fixed = 565 total (was 579) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 45s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10956/dev-support/hive-personality.sh
|
| git revision | master / 116bf0b |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: common ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10956/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Support ArrowOutputStream in LlapOutputFormatService
>
>
> Key: HIVE-19307
> URL: https://issues.apache.org/jira/browse/HIVE-19307
> Project: Hive
> Issue Type: Task
> Components: llap
>Reporter: Eric Wohlstadter
>Assignee: Eric Wohlstadter
>Priority: Major
> Attachments: HIVE-19307.3.patch, HIVE-19307.4.patch,
> HIVE-19307.5.patch, HIVE-19307.6.patch, HIVE-19307.7.patch, HIVE-19307.8.patch
>
>
> Support pushing arrow batches through
> org.apache.arrow.vector.ipc.ArrowOutputStream in LllapOutputFormatService.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19307) Support ArrowOutputStream in LlapOutputFormatService
[ https://issues.apache.org/jira/browse/HIVE-19307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474524#comment-16474524 ] Jason Dere commented on HIVE-19307: --- +1 from me, at least code-wise. This may have to wait for another clean ptest run after folks are done making some adjustments to the Hive PTest system > Support ArrowOutputStream in LlapOutputFormatService > > > Key: HIVE-19307 > URL: https://issues.apache.org/jira/browse/HIVE-19307 > Project: Hive > Issue Type: Task > Components: llap >Reporter: Eric Wohlstadter >Assignee: Eric Wohlstadter >Priority: Major > Attachments: HIVE-19307.3.patch, HIVE-19307.4.patch, > HIVE-19307.5.patch, HIVE-19307.6.patch, HIVE-19307.7.patch > > > Support pushing arrow batches through > org.apache.arrow.vector.ipc.ArrowOutputStream in LllapOutputFormatService. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19307) Support ArrowOutputStream in LlapOutputFormatService
[
https://issues.apache.org/jira/browse/HIVE-19307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16473890#comment-16473890
]
Hive QA commented on HIVE-19307:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12923226/HIVE-19307.7.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 40 failed/errored test(s), 14261 tests
executed
*Failed tests:*
{noformat}
TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=253)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=253)
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=253)
TestMinimrCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=94)
[infer_bucket_sort_num_buckets.q,infer_bucket_sort_reducers_power_two.q,parallel_orderby.q,bucket_num_reducers_acid.q,scriptfile1.q,infer_bucket_sort_map_operators.q,infer_bucket_sort_merge.q,root_dir_external_table.q,infer_bucket_sort_dyn_part.q,udf_using.q]
TestNonCatCallsWithCatalog - did not produce a TEST-*.xml file (likely timed
out) (batchId=217)
TestTezPerfCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=253)
TestTxnExIm - did not produce a TEST-*.xml file (likely timed out) (batchId=286)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[fouter_join_ppr]
(batchId=33)
org.apache.hadoop.hive.cli.TestMiniDruidKafkaCliDriver.org.apache.hadoop.hive.cli.TestMiniDruidKafkaCliDriver
(batchId=253)
org.apache.hadoop.hive.cli.TestMiniDruidKafkaCliDriver.testCliDriver[druidkafkamini_basic]
(batchId=253)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
(batchId=175)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[mm_bhif]
(batchId=157)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_orc_acidvec_part]
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_orc_vec_part_llap_io]
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[special_character_in_tabnames_1]
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_1]
(batchId=172)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_vector_dynpart_hashjoin_1]
(batchId=172)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[union_fast_stats]
(batchId=167)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[bucketizedhiveinputformat]
(batchId=183)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut
(batchId=225)
org.apache.hadoop.hive.metastore.TestOldSchema.testPartitionOps (batchId=217)
org.apache.hadoop.hive.metastore.TestStats.partitionedTableDeprecatedCalls
(batchId=211)
org.apache.hadoop.hive.metastore.client.TestRuntimeStats.testCleanup[Embedded]
(batchId=209)
org.apache.hadoop.hive.ql.TestAcidOnTez.testCtasTezUnion (batchId=228)
org.apache.hadoop.hive.ql.TestAcidOnTez.testNonStandardConversion01
(batchId=228)
org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1 (batchId=232)
org.apache.hive.beeline.TestBeeLineWithArgs.testQueryProgressParallel
(batchId=236)
org.apache.hive.jdbc.TestSSL.testSSLFetchHttp (batchId=239)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testMultipleTriggers2
(batchId=240)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomCreatedDynamicPartitions
(batchId=240)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomCreatedDynamicPartitionsMultiInsert
(batchId=240)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomCreatedDynamicPartitionsUnionAll
(batchId=240)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomCreatedFiles
(batchId=240)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomNonExistent
(batchId=240)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighBytesRead
(batchId=240)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighShuffleBytes
(batchId=240)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerVertexRawInputSplitsNoKill
(batchId=240)
org.apache.hive.service.server.TestInformationSchemaWithPrivilege.test
(batchId=238)
org.apache.hive.spark.client.rpc.TestRpc.testServerPort (batchId=304)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/10924/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/10924/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-10924/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
[jira] [Commented] (HIVE-19307) Support ArrowOutputStream in LlapOutputFormatService
[
https://issues.apache.org/jira/browse/HIVE-19307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16473853#comment-16473853
]
Hive QA commented on HIVE-19307:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
34s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
29s{color} | {color:blue} common in master has 62 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
47s{color} | {color:blue} ql in master has 2321 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
7s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} The patch common passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s{color} | {color:green} ql: The patch generated 0 new + 565 unchanged - 14
fixed = 565 total (was 579) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 34s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10924/dev-support/hive-personality.sh
|
| git revision | master / f327624 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: common ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10924/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Support ArrowOutputStream in LlapOutputFormatService
>
>
> Key: HIVE-19307
> URL: https://issues.apache.org/jira/browse/HIVE-19307
> Project: Hive
> Issue Type: Task
> Components: llap
>Reporter: Eric Wohlstadter
>Assignee: Eric Wohlstadter
>Priority: Major
> Attachments: HIVE-19307.3.patch, HIVE-19307.4.patch,
> HIVE-19307.5.patch, HIVE-19307.6.patch, HIVE-19307.7.patch
>
>
> Support pushing arrow batches through
> org.apache.arrow.vector.ipc.ArrowOutputStream in LllapOutputFormatService.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19307) Support ArrowOutputStream in LlapOutputFormatService
[
https://issues.apache.org/jira/browse/HIVE-19307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16473655#comment-16473655
]
Hive QA commented on HIVE-19307:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12923187/HIVE-19307.6.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 33 failed/errored test(s), 13595 tests
executed
*Failed tests:*
{noformat}
TestMinimrCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=94)
[infer_bucket_sort_num_buckets.q,infer_bucket_sort_reducers_power_two.q,parallel_orderby.q,bucket_num_reducers_acid.q,scriptfile1.q,infer_bucket_sort_map_operators.q,infer_bucket_sort_merge.q,root_dir_external_table.q,infer_bucket_sort_dyn_part.q,udf_using.q]
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=95)
[jira] [Commented] (HIVE-19307) Support ArrowOutputStream in LlapOutputFormatService
[
https://issues.apache.org/jira/browse/HIVE-19307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16473645#comment-16473645
]
Hive QA commented on HIVE-19307:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
42s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
33s{color} | {color:blue} common in master has 62 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
5s{color} | {color:blue} ql in master has 2321 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
8s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
20s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
46s{color} | {color:red} ql: The patch generated 3 new + 565 unchanged - 14
fixed = 568 total (was 579) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
9s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 27m 32s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10904/dev-support/hive-personality.sh
|
| git revision | master / d71032e |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10904/yetus/diff-checkstyle-ql.txt
|
| modules | C: common ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10904/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Support ArrowOutputStream in LlapOutputFormatService
>
>
> Key: HIVE-19307
> URL: https://issues.apache.org/jira/browse/HIVE-19307
> Project: Hive
> Issue Type: Task
> Components: llap
>Reporter: Eric Wohlstadter
>Assignee: Eric Wohlstadter
>Priority: Major
> Attachments: HIVE-19307.3.patch, HIVE-19307.4.patch,
> HIVE-19307.5.patch, HIVE-19307.6.patch
>
>
> Support pushing arrow batches through
> org.apache.arrow.vector.ipc.ArrowOutputStream in LllapOutputFormatService.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19307) Support ArrowOutputStream in LlapOutputFormatService
[
https://issues.apache.org/jira/browse/HIVE-19307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16473025#comment-16473025
]
Hive QA commented on HIVE-19307:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12922930/HIVE-19307.5.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 38 failed/errored test(s), 13578 tests
executed
*Failed tests:*
{noformat}
TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed
out) (batchId=247)
TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed
out) (batchId=247)
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=95)
[jira] [Commented] (HIVE-19307) Support ArrowOutputStream in LlapOutputFormatService
[
https://issues.apache.org/jira/browse/HIVE-19307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16473012#comment-16473012
]
Hive QA commented on HIVE-19307:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
43s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
30s{color} | {color:blue} common in master has 62 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
47s{color} | {color:blue} ql in master has 2321 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
6s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
41s{color} | {color:red} ql: The patch generated 14 new + 576 unchanged - 3
fixed = 590 total (was 579) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch 2 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 26m 0s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10851/dev-support/hive-personality.sh
|
| git revision | master / 615e71e |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10851/yetus/diff-checkstyle-ql.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10851/yetus/whitespace-tabs.txt
|
| modules | C: common ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10851/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Support ArrowOutputStream in LlapOutputFormatService
>
>
> Key: HIVE-19307
> URL: https://issues.apache.org/jira/browse/HIVE-19307
> Project: Hive
> Issue Type: Task
> Components: llap
>Reporter: Eric Wohlstadter
>Assignee: Eric Wohlstadter
>Priority: Major
> Attachments: HIVE-19307.3.patch, HIVE-19307.4.patch,
> HIVE-19307.5.patch
>
>
> Support pushing arrow batches through
> org.apache.arrow.vector.ipc.ArrowOutputStream in LllapOutputFormatService.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19307) Support ArrowOutputStream in LlapOutputFormatService
[ https://issues.apache.org/jira/browse/HIVE-19307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16469172#comment-16469172 ] Eric Wohlstadter commented on HIVE-19307: - [~jdere] That makes sense. I'll make that change in the next patch. > Support ArrowOutputStream in LlapOutputFormatService > > > Key: HIVE-19307 > URL: https://issues.apache.org/jira/browse/HIVE-19307 > Project: Hive > Issue Type: Task > Components: llap >Reporter: Eric Wohlstadter >Assignee: Eric Wohlstadter >Priority: Major > Attachments: HIVE-19307.3.patch, HIVE-19307.4.patch > > > Support pushing arrow batches through > org.apache.arrow.vector.ipc.ArrowOutputStream in LllapOutputFormatService. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19307) Support ArrowOutputStream in LlapOutputFormatService
[
https://issues.apache.org/jira/browse/HIVE-19307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16469160#comment-16469160
]
Jason Dere commented on HIVE-19307:
---
How about this comment?
{quote}
This is checking FileSinkDesc.isUsingThriftJDBCBinarySerDe(). I wonder if the
field/method in FileSinkDesc should be renamed to isUsingBatchingSerializer.
{quote}
isUsingThriftJDBCBinarySerDe would now be an inaccurate name for this field
since it indicates that the serde is either thriftJdbcBinary, or Arrow.
> Support ArrowOutputStream in LlapOutputFormatService
>
>
> Key: HIVE-19307
> URL: https://issues.apache.org/jira/browse/HIVE-19307
> Project: Hive
> Issue Type: Task
> Components: llap
>Reporter: Eric Wohlstadter
>Assignee: Eric Wohlstadter
>Priority: Major
> Attachments: HIVE-19307.3.patch, HIVE-19307.4.patch
>
>
> Support pushing arrow batches through
> org.apache.arrow.vector.ipc.ArrowOutputStream in LllapOutputFormatService.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19307) Support ArrowOutputStream in LlapOutputFormatService
[
https://issues.apache.org/jira/browse/HIVE-19307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16466520#comment-16466520
]
Eric Wohlstadter commented on HIVE-19307:
-
[~jdere]
I created HIVE-19445 to followup on:
{quote}I see now that the writeListener could be implemented in such a way as
to propagate a write error back to the writer (so we can possibly throw an
exception and fail the current operation rather than just log and ignore the
error). Plus on close I'm wondering if it is better just to wait for the close
future to complete so we can check the status.
{quote}
Will upload a new patch to fix the other suggestions.
> Support ArrowOutputStream in LlapOutputFormatService
>
>
> Key: HIVE-19307
> URL: https://issues.apache.org/jira/browse/HIVE-19307
> Project: Hive
> Issue Type: Task
> Components: llap
>Reporter: Eric Wohlstadter
>Assignee: Eric Wohlstadter
>Priority: Major
> Attachments: HIVE-19307.3.patch
>
>
> Support pushing arrow batches through
> org.apache.arrow.vector.ipc.ArrowOutputStream in LllapOutputFormatService.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19307) Support ArrowOutputStream in LlapOutputFormatService
[ https://issues.apache.org/jira/browse/HIVE-19307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461615#comment-16461615 ] Jason Dere commented on HIVE-19307: --- A few things: Nit: LlapOutputFormatService.java has a tab in the indentation, plus the indentation seems to be tab-aligned (too many spaces). Nit: SemanticAnalyzer.java has a couple of lines with trailing whitespace FileSinkOperator: This is checking FileSinkDesc.isUsingThriftJDBCBinarySerDe(). I wonder if the field/method in FileSinkDesc should be renamed to isUsingBatchingSerializer. WritableByteChannelAdapter: - Maybe some comments related to takeWriteResources() and the reason for that. - I know this is based on ChannelOutputStream, but actually I see now that the writeListener could be implemented in such a way as to propagate a write error back to the writer (so we can possibly throw an exception and fail the current operation rather than just log and ignore the error). Plus on close I'm wondering if it is better just to wait for the close future to complete so we can check the status. If you want we can take an item to address this later. > Support ArrowOutputStream in LlapOutputFormatService > > > Key: HIVE-19307 > URL: https://issues.apache.org/jira/browse/HIVE-19307 > Project: Hive > Issue Type: Task > Components: llap >Reporter: Eric Wohlstadter >Assignee: Eric Wohlstadter >Priority: Major > Attachments: HIVE-19307.3.patch > > > Support pushing arrow batches through > org.apache.arrow.vector.ipc.ArrowOutputStream in LllapOutputFormatService. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19307) Support ArrowOutputStream in LlapOutputFormatService
[ https://issues.apache.org/jira/browse/HIVE-19307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461515#comment-16461515 ] Eric Wohlstadter commented on HIVE-19307: - [~jdere] [~teddy.choi] Yes, it depends on HIVE-19306 which is still in progress. > Support ArrowOutputStream in LlapOutputFormatService > > > Key: HIVE-19307 > URL: https://issues.apache.org/jira/browse/HIVE-19307 > Project: Hive > Issue Type: Task > Components: llap >Reporter: Eric Wohlstadter >Assignee: Eric Wohlstadter >Priority: Major > Attachments: HIVE-19307.3.patch > > > Support pushing arrow batches through > org.apache.arrow.vector.ipc.ArrowOutputStream in LllapOutputFormatService. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19307) Support ArrowOutputStream in LlapOutputFormatService
[ https://issues.apache.org/jira/browse/HIVE-19307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461509#comment-16461509 ] Jason Dere commented on HIVE-19307: --- Looks like the patch failed to compile during ptests, is this patch dependent on other patches in progress? > Support ArrowOutputStream in LlapOutputFormatService > > > Key: HIVE-19307 > URL: https://issues.apache.org/jira/browse/HIVE-19307 > Project: Hive > Issue Type: Task > Components: llap >Reporter: Eric Wohlstadter >Assignee: Eric Wohlstadter >Priority: Major > Attachments: HIVE-19307.3.patch > > > Support pushing arrow batches through > org.apache.arrow.vector.ipc.ArrowOutputStream in LllapOutputFormatService. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19307) Support ArrowOutputStream in LlapOutputFormatService
[
https://issues.apache.org/jira/browse/HIVE-19307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461474#comment-16461474
]
Eric Wohlstadter commented on HIVE-19307:
-
TODO (before commit):
Need to verify that the line in {{WritableByteChannelAdapter}}
{code:java}
chc.writeAndFlush(Unpooled.wrappedBuffer(src))
{code}
does not leak memory.
> Support ArrowOutputStream in LlapOutputFormatService
>
>
> Key: HIVE-19307
> URL: https://issues.apache.org/jira/browse/HIVE-19307
> Project: Hive
> Issue Type: Task
> Components: llap
>Reporter: Eric Wohlstadter
>Assignee: Eric Wohlstadter
>Priority: Major
> Attachments: HIVE-19307.3.patch
>
>
> Support pushing arrow batches through
> org.apache.arrow.vector.ipc.ArrowOutputStream in LllapOutputFormatService.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19307) Support ArrowOutputStream in LlapOutputFormatService
[
https://issues.apache.org/jira/browse/HIVE-19307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16460513#comment-16460513
]
Hive QA commented on HIVE-19307:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12921528/HIVE-19307.3.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/10626/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/10626/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-10626/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2018-05-02 04:52:38.295
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-10626/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2018-05-02 04:52:38.299
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 4fd8f03 HIVE-19327: qroupby_rollup_empty.q fails for insert-only
transactional tables (Steve Yeom reviewed by Sergey Shelukhin, Prasanth
Jayachandran)
+ git clean -f -d
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 4fd8f03 HIVE-19327: qroupby_rollup_empty.q fails for insert-only
transactional tables (Steve Yeom reviewed by Sergey Shelukhin, Prasanth
Jayachandran)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2018-05-02 04:52:39.236
+ rm -rf ../yetus_PreCommit-HIVE-Build-10626
+ mkdir ../yetus_PreCommit-HIVE-Build-10626
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-10626
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-10626/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
Going to apply patch with: git apply -p0
/data/hiveptest/working/scratch/build.patch:96: trailing whitespace.
/data/hiveptest/working/scratch/build.patch:98: trailing whitespace.
return
(serdeClassName.equalsIgnoreCase(ThriftJDBCBinarySerDe.class.getName()) &&
warning: 2 lines add whitespace errors.
+ [[ maven == \m\a\v\e\n ]]
+ rm -rf /data/hiveptest/working/maven/org/apache/hive
+ mvn -B clean install -DskipTests -T 4 -q
-Dmaven.repo.local=/data/hiveptest/working/maven
protoc-jar: executing: [/tmp/protoc5427116293702843310.exe, --version]
libprotoc 2.5.0
protoc-jar: executing: [/tmp/protoc5427116293702843310.exe,
-I/data/hiveptest/working/apache-github-source-source/standalone-metastore/src/main/protobuf/org/apache/hadoop/hive/metastore,
--java_out=/data/hiveptest/working/apache-github-source-source/standalone-metastore/target/generated-sources,
/data/hiveptest/working/apache-github-source-source/standalone-metastore/src/main/protobuf/org/apache/hadoop/hive/metastore/metastore.proto]
ANTLR Parser Generator Version 3.5.2
Output file
/data/hiveptest/working/apache-github-source-source/standalone-metastore/target/generated-sources/org/apache/hadoop/hive/metastore/parser/FilterParser.java
does not exist: must build
/data/hiveptest/working/apache-github-source-source/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/parser/Filter.g
org/apache/hadoop/hive/metastore/parser/Filter.g
log4j:WARN No appenders could be found for logger (DataNucleus.Persistence).
log4j:WARN Please initialize the log4j system properly.
DataNucleus Enhancer (version 4.1.17) for API "JDO"
DataNucleus Enhancer completed with success for 40 classes.
ANTLR Parser Generator Version 3.5.2
Output file
/data/hiveptest/working/apache-github-source-source/ql/target/generated-sources/antlr3/org/apache/hadoop/hive/ql/parse/HiveLexer.java
does not exist: must build
[jira] [Commented] (HIVE-19307) Support ArrowOutputStream in LlapOutputFormatService
[
https://issues.apache.org/jira/browse/HIVE-19307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16460508#comment-16460508
]
Eric Wohlstadter commented on HIVE-19307:
-
[~jdere]
Updated patch with missing {{takeWriteResources}} in
{{WritableByteChannelAdapter.write}}
> Support ArrowOutputStream in LlapOutputFormatService
>
>
> Key: HIVE-19307
> URL: https://issues.apache.org/jira/browse/HIVE-19307
> Project: Hive
> Issue Type: Task
> Components: llap
>Reporter: Eric Wohlstadter
>Assignee: Eric Wohlstadter
>Priority: Major
> Attachments: HIVE-19307.3.patch
>
>
> Support pushing arrow batches through
> org.apache.arrow.vector.ipc.ArrowOutputStream in LllapOutputFormatService.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
