[jira] [Commented] (HIVE-16061) Some of console output is not printed to the beeline console
[
https://issues.apache.org/jira/browse/HIVE-16061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15903508#comment-15903508
]
Hive QA commented on HIVE-16061:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12857027/HIVE-16061.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 10 failed/errored test(s), 10335 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_if_expr]
(batchId=141)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[auto_join_without_localtask]
(batchId=96)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[auto_sortmerge_join_3]
(batchId=96)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[date_join1]
(batchId=96)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[groupby6_noskew]
(batchId=96)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[mapjoin_test_outer]
(batchId=96)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[merge2] (batchId=96)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[smb_mapjoin_11]
(batchId=96)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vectorization_9]
(batchId=96)
org.apache.hive.service.cli.operation.TestOperationLoggingLayout.testSwitchLogLayout
(batchId=215)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/4053/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/4053/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-4053/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 10 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12857027 - PreCommit-HIVE-Build
> Some of console output is not printed to the beeline console
>
>
> Key: HIVE-16061
> URL: https://issues.apache.org/jira/browse/HIVE-16061
> Project: Hive
> Issue Type: Bug
> Components: Logging
>Affects Versions: 2.1.1
>Reporter: Aihua Xu
>Assignee: Aihua Xu
> Attachments: HIVE-16061.1.patch, HIVE-16061.2.patch
>
>
> Run a hiveserver2 instance "hive --service hiveserver2".
> Then from another console, connect to hiveserver2 "beeline -u
> "jdbc:hive2://localhost:1"
> When you run a MR job like "select t1.key from src t1 join src t2 on
> t1.key=t2.key", some of the console logs like MR job info are not printed to
> the console while it just print to the hiveserver2 console.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-16061) Some of console output is not printed to the beeline console
[ https://issues.apache.org/jira/browse/HIVE-16061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15903235#comment-15903235 ] Aihua Xu commented on HIVE-16061: - patch-2: change LogHelper to output to OperationLog so the console output also output to the beeline client console. We still keep the output to server console since it's used by HiveCLI. > Some of console output is not printed to the beeline console > > > Key: HIVE-16061 > URL: https://issues.apache.org/jira/browse/HIVE-16061 > Project: Hive > Issue Type: Bug > Components: Logging >Affects Versions: 2.1.1 >Reporter: Aihua Xu >Assignee: Aihua Xu > Attachments: HIVE-16061.1.patch, HIVE-16061.2.patch > > > Run a hiveserver2 instance "hive --service hiveserver2". > Then from another console, connect to hiveserver2 "beeline -u > "jdbc:hive2://localhost:1" > When you run a MR job like "select t1.key from src t1 join src t2 on > t1.key=t2.key", some of the console logs like MR job info are not printed to > the console while it just print to the hiveserver2 console. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-16061) Some of console output is not printed to the beeline console
[
https://issues.apache.org/jira/browse/HIVE-16061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895604#comment-15895604
]
Hive QA commented on HIVE-16061:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12855969/HIVE-16061.1.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 48 failed/errored test(s), 10281 tests
executed
*Failed tests:*
{noformat}
TestCommandProcessorFactory - did not produce a TEST-*.xml file (likely timed
out) (batchId=272)
TestDbTxnManager - did not produce a TEST-*.xml file (likely timed out)
(batchId=272)
TestDummyTxnManager - did not produce a TEST-*.xml file (likely timed out)
(batchId=272)
TestHiveInputSplitComparator - did not produce a TEST-*.xml file (likely timed
out) (batchId=272)
TestIndexType - did not produce a TEST-*.xml file (likely timed out)
(batchId=272)
TestSplitFilter - did not produce a TEST-*.xml file (likely timed out)
(batchId=272)
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[escape_comments]
(batchId=229)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[stats_aggregator_error_1]
(batchId=68)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[stats_publisher_error_1]
(batchId=74)
org.apache.hadoop.hive.cli.TestContribNegativeCliDriver.testCliDriver[case_with_row_sequence]
(batchId=226)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_text_vec_table]
(batchId=147)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[avro_non_nullable_union]
(batchId=86)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[create_function_nonexistent_class]
(batchId=85)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[create_function_nonudf_class]
(batchId=85)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[create_unknown_genericudf]
(batchId=86)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[create_unknown_udf_udaf]
(batchId=86)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[dyn_part3]
(batchId=85)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[dyn_part_max]
(batchId=85)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[dyn_part_max_per_node]
(batchId=85)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[dynamic_partitions_with_whitelist]
(batchId=86)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[index_compact_entry_limit]
(batchId=85)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[index_compact_size_limit]
(batchId=85)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[ivyDownload]
(batchId=85)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[stats_aggregator_error_1]
(batchId=86)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[stats_aggregator_error_2]
(batchId=86)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[stats_publisher_error_1]
(batchId=86)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[stats_publisher_error_2]
(batchId=85)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[udf_function_does_not_implement_udf]
(batchId=86)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[udf_nonexistent_resource]
(batchId=85)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vector_between_in]
(batchId=119)
org.apache.hadoop.hive.cli.TestSparkNegativeCliDriver.org.apache.hadoop.hive.cli.TestSparkNegativeCliDriver
(batchId=231)
org.apache.hive.beeline.TestSchemaTool.testNestedScriptsForDerby (batchId=212)
org.apache.hive.beeline.TestSchemaTool.testNestedScriptsForMySQL (batchId=212)
org.apache.hive.beeline.TestSchemaTool.testNestedScriptsForOracle (batchId=212)
org.apache.hive.beeline.TestSchemaTool.testPostgresFilter (batchId=212)
org.apache.hive.beeline.TestSchemaTool.testSchemaInit (batchId=212)
org.apache.hive.beeline.TestSchemaTool.testSchemaInitDryRun (batchId=212)
org.apache.hive.beeline.TestSchemaTool.testSchemaUpgrade (batchId=212)
org.apache.hive.beeline.TestSchemaTool.testSchemaUpgradeDryRun (batchId=212)
org.apache.hive.beeline.TestSchemaTool.testScriptMultiRowComment (batchId=212)
org.apache.hive.beeline.TestSchemaTool.testScriptWithDelimiter (batchId=212)
org.apache.hive.beeline.TestSchemaTool.testScripts (batchId=212)
org.apache.hive.beeline.TestSchemaTool.testValidateLocations (batchId=212)
org.apache.hive.beeline.TestSchemaTool.testValidateNullValues (batchId=212)
org.apache.hive.beeline.TestSchemaTool.testValidateSchemaTables (batchId=212)
org.apache.hive.beeline.TestSchemaTool.testValidateSchemaVersions (batchId=212)
org.apache.hive.beeline.TestSchemaTool.testValidateSequences (batchId=212)
org.apache.hive.service.cli.operation.TestOperationLoggingLayout.testSwitchLogLayout
(batchId=214)
{noformat}
Test results:
[jira] [Commented] (HIVE-16061) Some of console output is not printed to the beeline console
[
https://issues.apache.org/jira/browse/HIVE-16061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895041#comment-15895041
]
Aihua Xu commented on HIVE-16061:
-
Verified with beeline. Here is the output in beeline now.
{noformat}
2017-03-03T16:15:16,081 INFO [main] org.apache.hadoop.hive.conf.HiveConf -
Found configuration file null
2017-03-03T16:15:17,432 WARN [main] org.apache.hadoop.hive.common.LogUtils -
hive-site.xml not found on CLASSPATH
2017-03-03 16:15:17 Starting to launch local task to process map join;
maximum memory = 932184064
2017-03-03 16:15:18 Dump the side-table for tag: 0 with group count: 309
into file:
file:/tmp/hive/anonymous/19c9aac1-1992-4061-b6ba-6eb4b9605656/hive_2017-03-03_16-15-12_037_6838673076291140467-1/-local-10004/HashTable-Stage-3/MapJoin-mapfile00--.hashtable
2017-03-03 16:15:18 Uploaded 1 File to:
file:/tmp/hive/anonymous/19c9aac1-1992-4061-b6ba-6eb4b9605656/hive_2017-03-03_16-15-12_037_6838673076291140467-1/-local-10004/HashTable-Stage-3/MapJoin-mapfile00--.hashtable
(7485 bytes)
2017-03-03 16:15:18 End of local task; Time Taken: 0.555 sec.
Execution completed successfully
MapredLocal task succeeded
Number of reduce tasks is set to 0 since there's no reduce operator
Starting Job = job_local1477220543_0001, Tracking URL = http://localhost:8080/
Kill Command =
/Users/axu/Documents/workspaces/tools/hadoop/hadoop-2.6.0/bin/hadoop job -kill
job_local1477220543_0001
Hadoop job information for Stage-3: number of mappers: 0; number of reducers: 0
2017-03-03 16:15:19,904 Stage-3 map = 0%, reduce = 0%
Ended Job = job_local1477220543_0001 with errors
Error during job, obtaining debugging information...
{noformat}
> Some of console output is not printed to the beeline console
>
>
> Key: HIVE-16061
> URL: https://issues.apache.org/jira/browse/HIVE-16061
> Project: Hive
> Issue Type: Bug
> Components: Logging
>Affects Versions: 2.1.1
>Reporter: Aihua Xu
>Assignee: Aihua Xu
> Attachments: HIVE-16061.1.patch
>
>
> Run a hiveserver2 instance "hive --service hiveserver2".
> Then from another console, connect to hiveserver2 "beeline -u
> "jdbc:hive2://localhost:1"
> When you run a MR job like "select t1.key from src t1 join src t2 on
> t1.key=t2.key", some of the console logs like MR job info are not printed to
> the console while it just print to the hiveserver2 console.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-16061) Some of console output is not printed to the beeline console
[ https://issues.apache.org/jira/browse/HIVE-16061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895036#comment-15895036 ] Aihua Xu commented on HIVE-16061: - The issue is: we are printing the log to server console while we need to print to OperationLog so it will pass to the client. Patch-1: create a ClientConsole class and uses the class when we need to pass the info to the client. > Some of console output is not printed to the beeline console > > > Key: HIVE-16061 > URL: https://issues.apache.org/jira/browse/HIVE-16061 > Project: Hive > Issue Type: Bug > Components: Logging >Affects Versions: 2.1.1 >Reporter: Aihua Xu >Assignee: Aihua Xu > Attachments: HIVE-16061.1.patch > > > Run a hiveserver2 instance "hive --service hiveserver2". > Then from another console, connect to hiveserver2 "beeline -u > "jdbc:hive2://localhost:1" > When you run a MR job like "select t1.key from src t1 join src t2 on > t1.key=t2.key", some of the console logs like MR job info are not printed to > the console while it just print to the hiveserver2 console. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-16061) Some of console output is not printed to the beeline console
[
https://issues.apache.org/jira/browse/HIVE-16061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15890274#comment-15890274
]
Aihua Xu commented on HIVE-16061:
-
[~muxin] Thanks for the info. Your analysis makes sense and the direction looks
good. Seems we are not printing some info to OperationLog so the client can't
query it.
Seems you are focusing on a different part of logging in JobDebugger. I tried
your patch, seems it's still not printing MR progress.
{noformat}
0: jdbc:hive2://localhost:1> select t1.key from src t1 join src t2 on
t1.key=t2.key limit 10;
WARNING: Hive-on-MR is deprecated in Hive 2 and may not be available in the
future versions. Consider using a different execution engine (i.e. tez, spark)
or using Hive 1.X releases.
+-+
| t1.key |
+-+
| 0 |
| 10 |
+-+
{noformat}
> Some of console output is not printed to the beeline console
>
>
> Key: HIVE-16061
> URL: https://issues.apache.org/jira/browse/HIVE-16061
> Project: Hive
> Issue Type: Bug
> Components: Logging
>Affects Versions: 2.1.1
>Reporter: Aihua Xu
>Assignee: Aihua Xu
>
> Run a hiveserver2 instance "hive --service hiveserver2".
> Then from another console, connect to hiveserver2 "beeline -u
> "jdbc:hive2://localhost:1"
> When you run a MR job like "select t1.key from src t1 join src t2 on
> t1.key=t2.key", some of the console logs like MR job info are not printed to
> the console while it just print to the hiveserver2 console.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-16061) Some of console output is not printed to the beeline console
[ https://issues.apache.org/jira/browse/HIVE-16061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15889215#comment-15889215 ] muxin commented on HIVE-16061: -- we met and resolved it in issue https://issues.apache.org/jira/browse/HIVE-15821 > Some of console output is not printed to the beeline console > > > Key: HIVE-16061 > URL: https://issues.apache.org/jira/browse/HIVE-16061 > Project: Hive > Issue Type: Bug > Components: Logging >Affects Versions: 2.1.1 >Reporter: Aihua Xu >Assignee: Aihua Xu > > Run a hiveserver2 instance "hive --service hiveserver2". > Then from another console, connect to hiveserver2 "beeline -u > "jdbc:hive2://localhost:1" > When you run a MR job like "select t1.key from src t1 join src t2 on > t1.key=t2.key", some of the console logs like MR job info are not printed to > the console while it just print to the hiveserver2 console. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
