[ 
https://issues.apache.org/jira/browse/PIG-5371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16725861#comment-16725861
 ] 

Adam Szita edited comment on PIG-5371 at 12/20/18 1:56 PM:
-----------------------------------------------------------

Hi [~abstractdog],

Yeah sorry that's a typo, indeed -Dtestcase should be used.

The test doesn't hang on my side, it finishes successfully in 9 minutes.
 Do you see which test method completes on your side and which one doesn't?

In the past when I faced with the hanging issue was due to my Mac's HDD had 
over 90% utilisation which some HDFS code in MiniCluster did not like


was (Author: szita):
Hi [~abstractdog],

Yeah sorry that's a typo, indeed -Dtestcase should be used.

The doesn't hang on my side, it finishes successfully in 9 minutes.
Do you see which test method completes on your side and which one doesn't?

In the past when I faced with the hanging issue was due to my Mac's HDD had 
over 90% utilisation which some HDFS code in MiniCluster did not like

> Hdfs bytes written assertions fail in TestPigRunner
> ---------------------------------------------------
>
>                 Key: PIG-5371
>                 URL: https://issues.apache.org/jira/browse/PIG-5371
>             Project: Pig
>          Issue Type: Bug
>            Reporter: Laszlo Bodor
>            Assignee: Laszlo Bodor
>            Priority: Major
>         Attachments: PIG-5371.01.patch, simpleTest.out
>
>
> Attached  [^simpleTest.out]. It seems like HDFS counter 'HDFS_BYTES_WRITTEN' 
> returns the byte count not only for the result of pig store operator, but it 
> includes the size of the jar files as well. The problem is this could change 
> very easily, so in my opinion the best would be to remove these assertions 
> from TestPigRunner as this is just causing intermittent and/or persistent 
> failures.
> The test class is for basic testing of PigRunner, and this is achieved well 
> enough without the asserts.
> {code}
> 2018-11-23 10:14:52,661 [IPC Server handler 5 on 54929] INFO  
> org.apache.hadoop.hdfs.StateChange - BLOCK* allocate blk_1073741827_1003, 
> replicas=127.0.0.1:54934, 127.0.0.1:54930, 127.0.0.1:54943 for 
> /tmp/temp-157262781/tmp-1057655772/automaton-1.11-8.jar
> ...
> 2018-11-23 10:14:52,735 [PacketResponder: 
> BP-26001448-10.200.50.195-1542964474138:blk_1073741827_1003, 
> type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=2:[127.0.0.1:54930, 
> 127.0.0.1:54943]] INFO  
> org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace - src: 
> /127.0.0.1:54978, dest: /127.0.0.1:54934, bytes: 176285, op: HDFS_WRITE, 
> cliID: DFSClient_NONMAPREDUCE_-1959727442_1, offset: 0, srvID: 
> 108c4000-1ae0-402e-82cf-bf403629c0f7, blockid: 
> BP-26001448-10.200.50.195-1542964474138:blk_1073741827_1003, duration(ns): 
> 57162859
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Reply via email to