[
https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17244908#comment-17244908
]
angerszhu commented on HDFS-14437:
--
Seems the Precondition is added later. I think we can ignore this
[
https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17244711#comment-17244711
]
Ravuri Sushma sree commented on HDFS-14437:
---
[~angerszhuuu]
I'm using 3.1.1 Hadoop version
[
https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17244428#comment-17244428
]
angerszhu commented on HDFS-14437:
--
[~Sushma_28]
No, don't meet this problem, can you show the detail
[
https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17244117#comment-17244117
]
Ravuri Sushma sree commented on HDFS-14437:
---
Thanks for the reply [~angerszhuuu].
Sounds great!
[
https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17243767#comment-17243767
]
angerszhu commented on HDFS-14437:
--
[~Sushma_28]
I have updated this patch in our product env for 20
[
https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17243763#comment-17243763
]
Ravuri Sushma sree commented on HDFS-14437:
---
I used the HDFS-14437.reproduction.patch provided
[
https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17240663#comment-17240663
]
Xiaoqiao He commented on HDFS-14437:
[~Sushma_28] Actually, I also meet the similar case (NameNode
[
https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17240493#comment-17240493
]
Ravuri Sushma sree commented on HDFS-14437:
---
Thank you everyone for the discussion here, I have
[
https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932246#comment-16932246
]
He Xiaoqiao commented on HDFS-14437:
[~daryn],[~jojochuang],[~linyiqun],[~xkrogen],[~elgoiri], are
[
https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928584#comment-16928584
]
feiwang commented on HDFS-14437:
[~linyiqun] Hi, yiqun, are you interested in this issue?
> Exception
[
https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928556#comment-16928556
]
angerszhu commented on HDFS-14437:
--
Hi watchers, after dig deep FSEditLog logic and try to reproduce
[
https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928141#comment-16928141
]
angerszhu commented on HDFS-14437:
--
[~daryn]
In current hadoop truck branch, still have this bug.
>
[
https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16832024#comment-16832024
]
Daryn Sharp commented on HDFS-14437:
Unless I'm overlooking new details, I explained on the cited
[
https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16826612#comment-16826612
]
angerszhu commented on HDFS-14437:
--
[~starphin]
it seem work not well when high concurrency. LOCK in
[
https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16826604#comment-16826604
]
star commented on HDFS-14437:
-
As [~kihwal] said in HDFS-10943 that '{{rollEditLog()}} does not and cannot
[
https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16824719#comment-16824719
]
angerszhu commented on HDFS-14437:
--
[~starphin]
Yesterday I omit a point that in the process table you
[
https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16823779#comment-16823779
]
angerszhu commented on HDFS-14437:
--
[~hexiaoqiao]
[angers@gmail.com|mailto:angers@gmail.com]
[
https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16823774#comment-16823774
]
He Xiaoqiao commented on HDFS-14437:
[~angerszhuuu] I would like to connect with you offline and
[
https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16823769#comment-16823769
]
angerszhu commented on HDFS-14437:
--
[~hadoopqa]
Hi can you test my pull request.
Thanks
> Exception
[
https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16823750#comment-16823750
]
star commented on HDFS-14437:
-
ok, Later I will take a view.
> Exception happened when rollEditLog expects
[
https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16823746#comment-16823746
]
angerszhu commented on HDFS-14437:
--
[~starphin]
You can see my pull request, two way both can solve
[
https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16823742#comment-16823742
]
star commented on HDFS-14437:
-
[~angerszhuuu] Right. lock state fixed.
{quote}But logAppend also need lock.
[
https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16823732#comment-16823732
]
angerszhu commented on HDFS-14437:
--
[~starphin]
But logAppend also need lock.
What you show is just
[
https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16823726#comment-16823726
]
star commented on HDFS-14437:
-
locked is object lock of FSEditLog.
> Exception happened when rollEditLog
[
https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16823717#comment-16823717
]
angerszhu commented on HDFS-14437:
--
[~starphin]
yeah , that is what i want to show .
you can see my
[
https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16823714#comment-16823714
]
star commented on HDFS-14437:
-
[~angerszhuuu], I think I've got that.
[
https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16823669#comment-16823669
]
angerszhu commented on HDFS-14437:
--
[~hexiaoqiao]
You can see that when other thread is also run
[
https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16823660#comment-16823660
]
He Xiaoqiao commented on HDFS-14437:
Thanks [~angerszhuuu] for your more information.
{quote}When
[
https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16822863#comment-16822863
]
angerszhu commented on HDFS-14437:
--
[~hexiaoqiao]
Look the table above your last comment. This
[
https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16822861#comment-16822861
]
angerszhu commented on HDFS-14437:
--
Hi [~hexiaoqiao]
you are right that seems some point is conflict of
[
https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16822856#comment-16822856
]
He Xiaoqiao commented on HDFS-14437:
[~angerszhuuu] sorry for my late response. I believe you try to
[
https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16822835#comment-16822835
]
angerszhu commented on HDFS-14437:
--
Auto Scheduler Sync 's step2(flush ) can process async
Call
[
https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16822096#comment-16822096
]
angerszhu commented on HDFS-14437:
--
[~hexiaoqiao]
add #waitForSyncToFinish before #endCurrentLogSegment
[
https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16821898#comment-16821898
]
angerszhu commented on HDFS-14437:
--
[~starphin]
Just add some code in
[
https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16821862#comment-16821862
]
star commented on HDFS-14437:
-
[~angerszhuuu], better upload your unit test code for others to reproduce the
[
https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16821826#comment-16821826
]
angerszhu commented on HDFS-14437:
--
[~hexiaoqiao] [~daryn] [~starphin] [~kihwal] [~arpitagarwal]
when
[
https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16821808#comment-16821808
]
angerszhu commented on HDFS-14437:
--
In the code of #logSync() this #wait
{code:java}
while (mytxid >
[
https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16821795#comment-16821795
]
angerszhu commented on HDFS-14437:
--
[~hexiaoqiao] [~daryn] [~starphin] [~kihwal] [~arpitagarwal]
[
https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16821767#comment-16821767
]
He Xiaoqiao commented on HDFS-14437:
[~angerszhuuu], would you like to supply detailed about
[
https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16821756#comment-16821756
]
angerszhu commented on HDFS-14437:
--
[~hexiaoqiao]
I just run it. And reappear it
{code:java}
[
https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16821755#comment-16821755
]
He Xiaoqiao commented on HDFS-14437:
[~angerszhuuu], I try to run unit test local and debug with
[
https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16821705#comment-16821705
]
angerszhu commented on HDFS-14437:
--
[~hexiaoqiao]
hi He Xiaoqiao, how do you run local test to
[
https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16821095#comment-16821095
]
angerszhu commented on HDFS-14437:
--
[~starphin]
Since #endCurrentLogSegment() have gain the lock of
[
https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16821092#comment-16821092
]
star commented on HDFS-14437:
-
[~angerszhuuu]. If fsLock ignored, the senario He Xiaoqiao listed could occur
[
https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16821040#comment-16821040
]
He Xiaoqiao commented on HDFS-14437:
[~angerszhuuu],Thanks for your correction, I will check it
[
https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16820974#comment-16820974
]
He Xiaoqiao commented on HDFS-14437:
Thanks [~starphin].
{quote}But the table you presented will
[
https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16820972#comment-16820972
]
angerszhu commented on HDFS-14437:
--
The scenario won't happen since the thread that call #rollEditLog
[
https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16820969#comment-16820969
]
angerszhu commented on HDFS-14437:
--
[~hexiaoqiao]
In my first description, I make something mistake.
[
https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16820953#comment-16820953
]
He Xiaoqiao commented on HDFS-14437:
[~angerszhuuu],[~starphin] please help to double check.
[
https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16820950#comment-16820950
]
star commented on HDFS-14437:
-
[~hexiaoqiao], agreed.
{quote}In my opinion, The following code segment in
[
https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16820942#comment-16820942
]
He Xiaoqiao commented on HDFS-14437:
Thanks [~angerszhuuu],[~starphin], It looks that root cause is
[
https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16820903#comment-16820903
]
angerszhu commented on HDFS-14437:
--
[~hexiaoqiao] I have seen some version of code, you use 2.7.1, it
[
https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16820901#comment-16820901
]
angerszhu commented on HDFS-14437:
--
[~hexiaoqiao]
What I'm saying is not very clear, but I found what
[
https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16820886#comment-16820886
]
star commented on HDFS-14437:
-
Thanks [~hexiaoqiao] . The table really make the issue clearer.
I've tracking
[
https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16820750#comment-16820750
]
He Xiaoqiao commented on HDFS-14437:
[~angerszhuuu], tracking code logic again, and do you mean the
[
https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16820742#comment-16820742
]
He Xiaoqiao commented on HDFS-14437:
[~angerszhuuu], Thanks for the ping. It is interesting dig. I
[
https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16820703#comment-16820703
]
angerszhu commented on HDFS-14437:
--
If some one can understand Chinese, can contact with me to chat
[
https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16820687#comment-16820687
]
angerszhu commented on HDFS-14437:
--
[~yzhangal] [~hexiaoqiao]
hi I have meet this problem too, can you
[
https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16820686#comment-16820686
]
angerszhu commented on HDFS-14437:
--
[~kihwal] [~daryn]
Hi both, can your look at my explanation about
59 matches
Mail list logo
