[jira] [Commented] (HBASE-21149) TestIncrementalBackupWithBulkLoad may fail due to file copy failure

2018-10-24 Thread Wei-Chiu Chuang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16662519#comment-16662519 ] Wei-Chiu Chuang commented on HBASE-21149: - Yes, Hadoop 2.7/2.8 shouldn't be affected. >

[jira] [Commented] (HBASE-21149) TestIncrementalBackupWithBulkLoad may fail due to file copy failure

2018-10-24 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16662514#comment-16662514 ] Ted Yu commented on HBASE-21149: The test would pass for hadoop 2 build: {code} 2.7.7 {code} I think

[jira] [Commented] (HBASE-21149) TestIncrementalBackupWithBulkLoad may fail due to file copy failure

2018-10-24 Thread Wei-Chiu Chuang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16662508#comment-16662508 ] Wei-Chiu Chuang commented on HBASE-21149: - Ted, should you disable the test for the time being?

[jira] [Commented] (HBASE-21149) TestIncrementalBackupWithBulkLoad may fail due to file copy failure

2018-10-24 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16662451#comment-16662451 ] Ted Yu commented on HBASE-21149: Logged HBASE-21381 for refguide work. >

[jira] [Commented] (HBASE-21149) TestIncrementalBackupWithBulkLoad may fail due to file copy failure

2018-10-19 Thread Sean Busbey (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657431#comment-16657431 ] Sean Busbey commented on HBASE-21149: - Given the severity here, probably worth either reopening this

[jira] [Commented] (HBASE-21149) TestIncrementalBackupWithBulkLoad may fail due to file copy failure

2018-10-19 Thread Vladimir Rodionov (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657426#comment-16657426 ] Vladimir Rodionov commented on HBASE-21149: --- I upped the priority. Seems critical issue, since

[jira] [Commented] (HBASE-21149) TestIncrementalBackupWithBulkLoad may fail due to file copy failure

2018-10-19 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657119#comment-16657119 ] Ted Yu commented on HBASE-21149: Once HADOOP-15850 is integrated, I will mark this dup of HADOOP-15850.

[jira] [Commented] (HBASE-21149) TestIncrementalBackupWithBulkLoad may fail due to file copy failure

2018-10-17 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16653816#comment-16653816 ] Ted Yu commented on HBASE-21149: >From feedback of two hadoop committers over HADOOP-15850, this seems

[jira] [Commented] (HBASE-21149) TestIncrementalBackupWithBulkLoad may fail due to file copy failure

2018-10-16 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16652171#comment-16652171 ] Ted Yu commented on HBASE-21149: Added more DEBUG log: {code} 2018-10-16 17:59:05,032 DEBUG

[jira] [Commented] (HBASE-21149) TestIncrementalBackupWithBulkLoad may fail due to file copy failure

2018-10-16 Thread Vladimir Rodionov (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16652062#comment-16652062 ] Vladimir Rodionov commented on HBASE-21149: --- {quote} The fix is to avoid multiple files in one

[jira] [Commented] (HBASE-21149) TestIncrementalBackupWithBulkLoad may fail due to file copy failure

2018-10-16 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16651961#comment-16651961 ] Ted Yu commented on HBASE-21149: Ran the test against hadoop 3.0.1 and got the same error: {code}

[jira] [Commented] (HBASE-21149) TestIncrementalBackupWithBulkLoad may fail due to file copy failure

2018-10-15 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16651020#comment-16651020 ] Ted Yu commented on HBASE-21149: See my comment above:

[jira] [Commented] (HBASE-21149) TestIncrementalBackupWithBulkLoad may fail due to file copy failure

2018-10-15 Thread Vladimir Rodionov (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16651018#comment-16651018 ] Vladimir Rodionov commented on HBASE-21149: --- {quote} Patch v2 breaks multiple source files

[jira] [Commented] (HBASE-21149) TestIncrementalBackupWithBulkLoad may fail due to file copy failure

2018-10-15 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16650444#comment-16650444 ] Hadoop QA commented on HBASE-21149: --- | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote ||

[jira] [Commented] (HBASE-21149) TestIncrementalBackupWithBulkLoad may fail due to file copy failure

2018-10-15 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16650327#comment-16650327 ] Ted Yu commented on HBASE-21149: bq. kicked off distcp while some of the source files were being

[jira] [Commented] (HBASE-21149) TestIncrementalBackupWithBulkLoad may fail due to file copy failure

2018-10-15 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16650324#comment-16650324 ] Ted Yu commented on HBASE-21149: I tried to plugin custom OutputFormat : {code} +@Override +

[jira] [Commented] (HBASE-21149) TestIncrementalBackupWithBulkLoad may fail due to file copy failure

2018-10-15 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16650163#comment-16650163 ] Steve Loughran commented on HBASE-21149: I wouldn't blame distcp here, yet. This hints of a race

[jira] [Commented] (HBASE-21149) TestIncrementalBackupWithBulkLoad may fail due to file copy failure

2018-10-14 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649617#comment-16649617 ] Ted Yu commented on HBASE-21149: Patch v2 breaks multiple source files into multiple activations of

[jira] [Commented] (HBASE-21149) TestIncrementalBackupWithBulkLoad may fail due to file copy failure

2018-10-14 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649606#comment-16649606 ] Ted Yu commented on HBASE-21149: CopyCommitter#concatFileChunks would throw the following exception when

[jira] [Commented] (HBASE-21149) TestIncrementalBackupWithBulkLoad may fail due to file copy failure

2018-10-08 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642620#comment-16642620 ] Ted Yu commented on HBASE-21149: It seems the Warning was due to check during concatenation phase of

[jira] [Commented] (HBASE-21149) TestIncrementalBackupWithBulkLoad may fail due to file copy failure

2018-10-08 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642469#comment-16642469 ] Ted Yu commented on HBASE-21149: See the attached test output above. >

[jira] [Commented] (HBASE-21149) TestIncrementalBackupWithBulkLoad may fail due to file copy failure

2018-10-08 Thread Vladimir Rodionov (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642460#comment-16642460 ] Vladimir Rodionov commented on HBASE-21149: --- [~te...@apache.org], any chances to get full log

[jira] [Commented] (HBASE-21149) TestIncrementalBackupWithBulkLoad may fail due to file copy failure

2018-10-08 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642312#comment-16642312 ] Ted Yu commented on HBASE-21149: I reproduced test error running against hadoop 3.1.1 : {code} [ERROR]

[jira] [Commented] (HBASE-21149) TestIncrementalBackupWithBulkLoad may fail due to file copy failure

2018-09-21 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623691#comment-16623691 ] Ted Yu commented on HBASE-21149: >From

[jira] [Commented] (HBASE-21149) TestIncrementalBackupWithBulkLoad may fail due to file copy failure

2018-09-07 Thread Vladimir Rodionov (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16607459#comment-16607459 ] Vladimir Rodionov commented on HBASE-21149: --- We can configure only backup tests

[jira] [Commented] (HBASE-21149) TestIncrementalBackupWithBulkLoad may fail due to file copy failure

2018-09-07 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16607321#comment-16607321 ] Ted Yu commented on HBASE-21149: For TestIncrementalBackupWithBulkLoad, we can keep the incremental

[jira] [Commented] (HBASE-21149) TestIncrementalBackupWithBulkLoad may fail due to file copy failure

2018-09-07 Thread Sean Busbey (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16606742#comment-16606742 ] Sean Busbey commented on HBASE-21149: - {quote} We need to reuse JVM to shorten execution time.

[jira] [Commented] (HBASE-21149) TestIncrementalBackupWithBulkLoad may fail due to file copy failure

2018-09-06 Thread Vladimir Rodionov (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16606485#comment-16606485 ] Vladimir Rodionov commented on HBASE-21149: --- Every test class starts cluster, because surefire

[jira] [Commented] (HBASE-21149) TestIncrementalBackupWithBulkLoad may fail due to file copy failure

2018-09-05 Thread Vladimir Rodionov (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605134#comment-16605134 ] Vladimir Rodionov commented on HBASE-21149: --- Yes, it seems that the reason for a failure is

[jira] [Commented] (HBASE-21149) TestIncrementalBackupWithBulkLoad may fail due to file copy failure

2018-09-04 Thread Vladimir Rodionov (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16603479#comment-16603479 ] Vladimir Rodionov commented on HBASE-21149: --- I removed two steps in a test (incr backup and

[jira] [Commented] (HBASE-21149) TestIncrementalBackupWithBulkLoad may fail due to file copy failure

2018-09-04 Thread Vladimir Rodionov (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16603468#comment-16603468 ] Vladimir Rodionov commented on HBASE-21149: --- It takes 235.472 sec on my MBP. >

[jira] [Commented] (HBASE-21149) TestIncrementalBackupWithBulkLoad may fail due to file copy failure

2018-09-04 Thread Vladimir Rodionov (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16603377#comment-16603377 ] Vladimir Rodionov commented on HBASE-21149: --- [~te...@apache.org], this looks like test