[jira] [Commented] (HIVE-8910) Refactoring of PassThroughOutputFormat
[ https://issues.apache.org/jira/browse/HIVE-8910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14222050#comment-14222050 ] Hive QA commented on HIVE-8910: --- {color:green}Overall{color}: +1 all checks pass Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12682806/HIVE-8910.3.patch.txt {color:green}SUCCESS:{color} +1 tests passed Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1865/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1865/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-1865/ Messages: {noformat} Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase {noformat} This message is automatically generated. ATTACHMENT ID: 12682806 - PreCommit-HIVE-TRUNK-Build Refactoring of PassThroughOutputFormat --- Key: HIVE-8910 URL: https://issues.apache.org/jira/browse/HIVE-8910 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Navis Assignee: Navis Priority: Minor Attachments: HIVE-8910.1.patch.txt, HIVE-8910.2.patch.txt, HIVE-8910.3.patch.txt It's overly complicated just for doing simple wrapping of output format. Before things get more worse, we should refactor this codes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8910) Refactoring of PassThroughOutputFormat
[ https://issues.apache.org/jira/browse/HIVE-8910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14219500#comment-14219500 ] Hive QA commented on HIVE-8910: --- {color:red}Overall{color}: -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12682546/HIVE-8910.2.patch.txt {color:red}ERROR:{color} -1 due to 14 failed/errored test(s), 6651 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_create_view_partitioned org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_external_table_ppd org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_binary_external_table_queries org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_binary_map_queries org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_binary_map_queries_prefix org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_binary_storage_queries org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_joins org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_ppd_join org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_ppd_key_range org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_pushdown org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_queries org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_single_sourced_multi_insert org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_timestamp org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_ppd_key_ranges {noformat} Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1848/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1848/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-1848/ Messages: {noformat} Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 14 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12682546 - PreCommit-HIVE-TRUNK-Build Refactoring of PassThroughOutputFormat --- Key: HIVE-8910 URL: https://issues.apache.org/jira/browse/HIVE-8910 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Navis Assignee: Navis Priority: Minor Attachments: HIVE-8910.1.patch.txt, HIVE-8910.2.patch.txt It's overly complicated just for doing simple wrapping of output format. Before things get more worse, we should refactor this codes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8910) Refactoring of PassThroughOutputFormat
[ https://issues.apache.org/jira/browse/HIVE-8910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14219958#comment-14219958 ] Sushanth Sowmyan commented on HIVE-8910: Per the qa test fails, looks like there is still an order of instantiation or copying job properties to conf issue to solve : {noformat} Begin query: hbase_binary_external_table_queries.q java.lang.IllegalArgumentException: Must specify table name at org.apache.hadoop.hbase.mapreduce.TableOutputFormat.setConf(TableOutputFormat.java:188) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:73) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133) at org.apache.hadoop.hive.ql.io.HiveFileFormatUtils.getHiveOutputFormat(HiveFileFormatUtils.java:272) at org.apache.hadoop.hive.ql.io.HiveFileFormatUtils.getHiveOutputFormat(HiveFileFormatUtils.java:267) at org.apache.hadoop.hive.ql.exec.Utilities.createDummyFileForEmptyPartition(Utilities.java:3354) at org.apache.hadoop.hive.ql.exec.Utilities.getInputPaths(Utilities.java:3285) at org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(ExecDriver.java:371) at org.apache.hadoop.hive.ql.exec.mr.MapRedTask.execute(MapRedTask.java:137) at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160) at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85) at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1644) at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1404) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1217) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1044) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1034) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:199) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:151) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:362) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:297) at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:849) at org.apache.hadoop.hive.cli.TestHBaseCliDriver.runTest(TestHBaseCliDriver.java:112) {noformat} This can happen when we instantiate an OutputFormat associated with a storage driver without doing the set up for it first, via StorageHandler.configureOutputJobProperties or StorageHandler.configureInputJobProperties, and setting those job properties in the conf provided to the outputformat. Refactoring of PassThroughOutputFormat --- Key: HIVE-8910 URL: https://issues.apache.org/jira/browse/HIVE-8910 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Navis Assignee: Navis Priority: Minor Attachments: HIVE-8910.1.patch.txt, HIVE-8910.2.patch.txt It's overly complicated just for doing simple wrapping of output format. Before things get more worse, we should refactor this codes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8910) Refactoring of PassThroughOutputFormat
[ https://issues.apache.org/jira/browse/HIVE-8910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14217659#comment-14217659 ] Hive QA commented on HIVE-8910: --- {color:red}Overall{color}: -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12682367/HIVE-8910.1.patch.txt {color:red}ERROR:{color} -1 due to 54 failed/errored test(s), 6648 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_authorization_index org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_database_drop org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_fileformat_mix org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_auth org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_auto org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_auto_empty org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_auto_file_format org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_auto_mult_tables org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_auto_mult_tables_compact org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_auto_multiple org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_auto_partitioned org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_auto_self_join org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_auto_unused org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_auto_update org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_bitmap org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_bitmap1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_bitmap2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_bitmap_auto_partitioned org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_bitmap_compression org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_bitmap_rc org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_compact org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_compact_1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_compact_2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_compact_3 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_compact_binary_search org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_compression org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_creation org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_serde org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_skewtable org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_stale org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_stale_partitioned org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_input45 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join13 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat3 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat4 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat5 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat6 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat8 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_show_indexes_edge_cases org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_show_indexes_syntax org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_view org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorization_short_regress org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorization_short_regress org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_index_bitmap3 org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_index_bitmap_auto org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_ql_rewrite_gbtoidx org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_schemeAuthority org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_create_insert_outputformat org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_fileformat_void_output org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_index_compact_entry_limit org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_index_compact_size_limit org.apache.hive.hcatalog.streaming.TestStreaming.testTransactionBatchCommit_Json {noformat} Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1841/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1841/console Test logs:
[jira] [Commented] (HIVE-8910) Refactoring of PassThroughOutputFormat
[ https://issues.apache.org/jira/browse/HIVE-8910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14218046#comment-14218046 ] Ashutosh Chauhan commented on HIVE-8910: I agree, its more complicated than it needs to be. Lets simplify! Refactoring of PassThroughOutputFormat --- Key: HIVE-8910 URL: https://issues.apache.org/jira/browse/HIVE-8910 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Navis Assignee: Navis Priority: Minor Attachments: HIVE-8910.1.patch.txt It's overly complicated just for doing simple wrapping of output format. Before things get more worse, we should refactor this codes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8910) Refactoring of PassThroughOutputFormat
[ https://issues.apache.org/jira/browse/HIVE-8910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14218867#comment-14218867 ] Ashutosh Chauhan commented on HIVE-8910: [~navis] Can you create RB entry for this ? Refactoring of PassThroughOutputFormat --- Key: HIVE-8910 URL: https://issues.apache.org/jira/browse/HIVE-8910 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Navis Assignee: Navis Priority: Minor Attachments: HIVE-8910.1.patch.txt, HIVE-8910.2.patch.txt It's overly complicated just for doing simple wrapping of output format. Before things get more worse, we should refactor this codes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8910) Refactoring of PassThroughOutputFormat
[ https://issues.apache.org/jira/browse/HIVE-8910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14218894#comment-14218894 ] Ashutosh Chauhan commented on HIVE-8910: +1 pending tests. cc: [~sushanth] Refactoring of PassThroughOutputFormat --- Key: HIVE-8910 URL: https://issues.apache.org/jira/browse/HIVE-8910 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Navis Assignee: Navis Priority: Minor Attachments: HIVE-8910.1.patch.txt, HIVE-8910.2.patch.txt It's overly complicated just for doing simple wrapping of output format. Before things get more worse, we should refactor this codes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8910) Refactoring of PassThroughOutputFormat
[ https://issues.apache.org/jira/browse/HIVE-8910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14218913#comment-14218913 ] Sushanth Sowmyan commented on HIVE-8910: This patch is beautiful, and solves so many issues, it brings tears to my eye. Thank you so much, Navis! +1 and a Bravo! Refactoring of PassThroughOutputFormat --- Key: HIVE-8910 URL: https://issues.apache.org/jira/browse/HIVE-8910 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Navis Assignee: Navis Priority: Minor Attachments: HIVE-8910.1.patch.txt, HIVE-8910.2.patch.txt It's overly complicated just for doing simple wrapping of output format. Before things get more worse, we should refactor this codes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)