[
https://issues.apache.org/jira/browse/HIVE-23754?focusedWorklogId=827702&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-827702
]
ASF GitHub Bot logged work on HIVE-23754:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 21/Nov/22 17:52
Start Date: 21/Nov/22 17:52
Worklog Time Spent: 10m
Work Description: difin commented on PR #3669:
URL: https://github.com/apache/hive/pull/3669#issuecomment-1322440527
> looks good, I'm always more than happy to see debugging features coming
into the code can you paste here a log snippet that becomes available with the
LoggingHandler?
[after -
hive.log](https://github.com/apache/hive/files/10059245/after.-.hive.log)
[before -
hive.log](https://github.com/apache/hive/files/10059246/before.-.hive.log)
Attached hive.log before and after adding Netty LoggingHandler.
Here is one of the snippets with the LoggingHandler:
`2022-11-21T09:14:00,478 INFO [TezTR-823713_1_2_2_0_0]
tez.ReduceRecordProcessor: Waiting for ShuffleInputs to become ready
2022-11-21T09:14:00,490 DEBUG [ShuffleHandler Netty Worker #2]
logging.LoggingHandler: [id: 0xee69f038, L:/10.0.0.75:52465 -
R:/10.0.0.75:52488] REGISTERED
2022-11-21T09:14:00,490 DEBUG [ShuffleHandler Netty Worker #2]
logging.LoggingHandler: [id: 0xee69f038, L:/10.0.0.75:52465 -
R:/10.0.0.75:52488] ACTIVE
2022-11-21T09:14:00,490 DEBUG [ShuffleHandler Netty Worker #3]
logging.LoggingHandler: [id: 0x6def8356, L:/10.0.0.75:52465 -
R:/10.0.0.75:52489] REGISTERED
2022-11-21T09:14:00,491 DEBUG [ShuffleHandler Netty Worker #3]
logging.LoggingHandler: [id: 0x6def8356, L:/10.0.0.75:52465 -
R:/10.0.0.75:52489] ACTIVE
2022-11-21T09:14:00,495 DEBUG [ShuffleHandler Netty Worker #2]
logging.LoggingHandler: [id: 0xee69f038, L:/10.0.0.75:52465 -
R:/10.0.0.75:52488] READ: 329B
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 47 45 54 20 2f 6d 61 70 4f 75 74 70 75 74 3f 6a |GET /mapOutput?j|
|00000010| 6f 62 3d 6a 6f 62 5f 31 36 36 39 30 35 30 38 32 |ob=job_166905082|
|00000020| 33 37 31 33 5f 30 30 30 31 26 64 61 67 3d 32 26 |3713_0001&dag=2&|
|00000030| 72 65 64 75 63 65 3d 30 26 6d 61 70 3d 61 74 74 |reduce=0&map=att|
|00000040| 65 6d 70 74 5f 31 36 36 39 30 35 30 38 32 33 37 |empt_16690508237|
|00000050| 31 33 5f 30 30 30 31 5f 32 5f 30 30 5f 30 30 30 |13_0001_2_00_000|
|00000060| 30 30 30 5f 30 5f 31 30 30 31 31 20 48 54 54 50 |000_0_10011 HTTP|
|00000070| 2f 31 2e 31 0d 0a 55 72 6c 48 61 73 68 3a 20 78 |/1.1..UrlHash: x|
|00000080| 61 74 6c 33 71 55 37 2b 4b 53 43 38 74 62 49 77 |atl3qU7+KSC8tbIw|
|00000090| 78 69 56 36 30 50 59 44 2b 34 3d 0d 0a 6e 61 6d |xiV60PYD+4=..nam|
|000000a0| 65 3a 20 6d 61 70 72 65 64 75 63 65 0d 0a 76 65 |e: mapreduce..ve|
|000000b0| 72 73 69 6f 6e 3a 20 31 2e 30 2e 30 0d 0a 55 73 |rsion: 1.0.0..Us|
|000000c0| 65 72 2d 41 67 65 6e 74 3a 20 4a 61 76 61 2f 31 |er-Agent: Java/1|
|000000d0| 2e 38 2e 30 5f 32 39 32 0d 0a 48 6f 73 74 3a 20 |.8.0_292..Host: |
|000000e0| 31 30 2e 30 2e 30 2e 37 35 3a 35 32 34 36 35 0d |10.0.0.75:52465.|
|000000f0| 0a 41 63 63 65 70 74 3a 20 74 65 78 74 2f 68 74 |.Accept: text/ht|
|00000100| 6d 6c 2c 20 69 6d 61 67 65 2f 67 69 66 2c 20 69 |ml, image/gif, i|
|00000110| 6d 61 67 65 2f 6a 70 65 67 2c 20 2a 3b 20 71 3d |mage/jpeg, *; q=|
|00000120| 2e 32 2c 20 2a 2f 2a 3b 20 71 3d 2e 32 0d 0a 43 |.2, */*; q=.2..C|
|00000130| 6f 6e 6e 65 63 74 69 6f 6e 3a 20 6b 65 65 70 2d |onnection: keep-|
|00000140| 61 6c 69 76 65 0d 0a 0d 0a |alive.... |
+--------+-------------------------------------------------+----------------+
2022-11-21T09:14:00,495 DEBUG [ShuffleHandler Netty Worker #3]
logging.LoggingHandler: [id: 0x6def8356, L:/10.0.0.75:52465 -
R:/10.0.0.75:52489] READ: 329B`
Issue Time Tracking
-------------------
Worklog Id: (was: 827702)
Time Spent: 1.5h (was: 1h 20m)
> LLAP: Add LoggingHandler in ShuffleHandler pipeline for better debuggability
> ----------------------------------------------------------------------------
>
> Key: HIVE-23754
> URL: https://issues.apache.org/jira/browse/HIVE-23754
> Project: Hive
> Issue Type: Improvement
> Environment:
>
> Reporter: Rajesh Balamohan
> Assignee: Dmitriy Fingerman
> Priority: Major
> Labels: pull-request-available
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> [https://github.com/apache/hive/blob/master/llap-server/src/java/org/apache/hadoop/hive/llap/shufflehandler/ShuffleHandler.java#L616]
>
> For corner case debugging, it would be helpful to understand when netty
> processed OPEN/BOUND/CLOSE/RECEIVED/CONNECTED events along with payload
> details.
> Adding "LoggingHandler" in ChannelPipeline mode can help in debugging.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
