[jira] [Updated] (HBASE-21457) BackupUtils#getWALFilesOlderThan refers to wrong FileSystem

2018-11-08 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-21457: --- Attachment: 21457.v3.txt > BackupUtils#getWALFilesOlderThan refers to wrong FileSystem >

[jira] [Commented] (HBASE-21246) Introduce WALIdentity interface

2018-11-08 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16680549#comment-16680549 ] Ted Yu commented on HBASE-21246: Looking at compilation errors for patch v22: 321 lines in the

[jira] [Commented] (HBASE-21457) BackupUtils#getWALFilesOlderThan refers to wrong FileSystem

2018-11-08 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16680531#comment-16680531 ] Ted Yu commented on HBASE-21457: The test passed during local run. Tests run: 1, Failures: 0, Errors:

[jira] [Commented] (HBASE-21457) BackupUtils#getWALFilesOlderThan refers to wrong FileSystem

2018-11-08 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16680500#comment-16680500 ] Ted Yu commented on HBASE-21457: I tried to retrieve output for the failed test but it looks like

[jira] [Commented] (HBASE-21457) BackupUtils#getWALFilesOlderThan refers to wrong FileSystem

2018-11-08 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16680435#comment-16680435 ] Ted Yu commented on HBASE-21457: I did a search in hbase-backup module where we retrieve FileSystem.

[jira] [Updated] (HBASE-21457) BackupUtils#getWALFilesOlderThan refers to wrong FileSystem

2018-11-08 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-21457: --- Attachment: 21457.v3.txt > BackupUtils#getWALFilesOlderThan refers to wrong FileSystem >

[jira] [Created] (HADOOP-15910) Javadoc for LdapAuthenticationHandler#ENABLE_START_TLS is wrong

2018-11-08 Thread Ted Yu (JIRA)
Ted Yu created HADOOP-15910: --- Summary: Javadoc for LdapAuthenticationHandler#ENABLE_START_TLS is wrong Key: HADOOP-15910 URL: https://issues.apache.org/jira/browse/HADOOP-15910 Project: Hadoop Common

[jira] [Updated] (HBASE-21457) BackupUtils#getWALFilesOlderThan refers to wrong FileSystem

2018-11-08 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-21457: --- Attachment: (was: 21457.v3.txt) > BackupUtils#getWALFilesOlderThan refers to wrong FileSystem >

[jira] [Updated] (HBASE-21457) BackupUtils#getWALFilesOlderThan refers to wrong FileSystem

2018-11-08 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-21457: --- Attachment: 21457.v3.txt > BackupUtils#getWALFilesOlderThan refers to wrong FileSystem >

[jira] [Commented] (HBASE-21457) BackupUtils#getWALFilesOlderThan refers to wrong FileSystem

2018-11-08 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16680380#comment-16680380 ] Ted Yu commented on HBASE-21457: Patch v3 modifies TestBackupBase to use second hdfs cluster for WAL

[jira] [Updated] (HBASE-21439) StochasticLoadBalancer RegionLoads aren’t being used in RegionLoad cost functions

2018-11-08 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-21439: --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0 Status: Resolved (was:

[jira] [Updated] (HBASE-21246) Introduce WALIdentity interface

2018-11-08 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-21246: --- Attachment: 21246.26.txt > Introduce WALIdentity interface > --- > >

[jira] [Updated] (HBASE-21457) BackupUtils#getWALFilesOlderThan refers to wrong FileSystem

2018-11-08 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-21457: --- Attachment: 21457.v2.txt > BackupUtils#getWALFilesOlderThan refers to wrong FileSystem >

[jira] [Created] (HADOOP-15910) Javadoc for LdapAuthenticationHandler#ENABLE_START_TLS is wrong

2018-11-08 Thread Ted Yu (JIRA)
Ted Yu created HADOOP-15910: --- Summary: Javadoc for LdapAuthenticationHandler#ENABLE_START_TLS is wrong Key: HADOOP-15910 URL: https://issues.apache.org/jira/browse/HADOOP-15910 Project: Hadoop Common

[jira] [Updated] (HBASE-21457) BackupUtils#getWALFilesOlderThan refers to wrong FileSystem

2018-11-08 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-21457: --- Status: Patch Available (was: Open) > BackupUtils#getWALFilesOlderThan refers to wrong FileSystem >

[jira] [Assigned] (HBASE-21457) BackupUtils#getWALFilesOlderThan refers to wrong FileSystem

2018-11-08 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu reassigned HBASE-21457: -- Assignee: Ted Yu > BackupUtils#getWALFilesOlderThan refers to wrong FileSystem >

[jira] [Created] (HBASE-21457) BackupUtils#getWALFilesOlderThan refers to wrong FileSystem

2018-11-08 Thread Ted Yu (JIRA)
Ted Yu created HBASE-21457: -- Summary: BackupUtils#getWALFilesOlderThan refers to wrong FileSystem Key: HBASE-21457 URL: https://issues.apache.org/jira/browse/HBASE-21457 Project: HBase Issue Type:

[jira] [Created] (HBASE-21457) BackupUtils#getWALFilesOlderThan refers to wrong FileSystem

2018-11-08 Thread Ted Yu (JIRA)
Ted Yu created HBASE-21457: -- Summary: BackupUtils#getWALFilesOlderThan refers to wrong FileSystem Key: HBASE-21457 URL: https://issues.apache.org/jira/browse/HBASE-21457 Project: HBase Issue Type:

[jira] [Updated] (HBASE-21457) BackupUtils#getWALFilesOlderThan refers to wrong FileSystem

2018-11-08 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-21457: --- Attachment: 21457.v1.txt > BackupUtils#getWALFilesOlderThan refers to wrong FileSystem >

[jira] [Commented] (HBASE-21457) BackupUtils#getWALFilesOlderThan refers to wrong FileSystem

2018-11-08 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16680048#comment-16680048 ] Ted Yu commented on HBASE-21457: bq. can you replace filesystem parameter fs with walFs in the lines

[jira] [Commented] (HBASE-21439) StochasticLoadBalancer RegionLoads aren’t being used in RegionLoad cost functions

2018-11-07 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16678766#comment-16678766 ] Ted Yu commented on HBASE-21439: StochasticLoadBalancer tests passed. lgtm, pending QA. >

[jira] [Commented] (HBASE-21387) Race condition surrounding in progress snapshot handling in snapshot cache leads to loss of snapshot files

2018-11-07 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16678531#comment-16678531 ] Ted Yu commented on HBASE-21387: Here is a brief summary of the approaches I tried, with most recent

[jira] [Updated] (HBASE-21387) Race condition surrounding in progress snapshot handling in snapshot cache leads to loss of snapshot files

2018-11-07 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-21387: --- Description: During recent report from customer where ExportSnapshot failed: {code} 2018-10-09 18:54:32,559

[jira] [Commented] (HBASE-21387) Race condition surrounding in progress snapshot handling in snapshot cache leads to loss of snapshot files

2018-11-07 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16678473#comment-16678473 ] Ted Yu commented on HBASE-21387: [~openinx][~Apache9][~elserj] : Please take a look at 21387.v9.txt

[jira] [Updated] (HBASE-21246) Introduce WALIdentity interface

2018-11-06 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-21246: --- Attachment: 21246.25.txt > Introduce WALIdentity interface > --- > >

[jira] [Commented] (HBASE-20952) Re-visit the WAL API

2018-11-06 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-20952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16677639#comment-16677639 ] Ted Yu commented on HBASE-20952: >From

[jira] [Updated] (HBASE-21387) Race condition surrounding in progress snapshot handling in snapshot cache leads to loss of snapshot files

2018-11-06 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-21387: --- Status: Patch Available (was: Open) > Race condition surrounding in progress snapshot handling in snapshot

[jira] [Updated] (HBASE-21387) Race condition surrounding in progress snapshot handling in snapshot cache leads to loss of snapshot files

2018-11-06 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-21387: --- Status: Open (was: Patch Available) > Race condition surrounding in progress snapshot handling in snapshot

[jira] [Updated] (HBASE-21387) Race condition surrounding in progress snapshot handling in snapshot cache leads to loss of snapshot files

2018-11-06 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-21387: --- Status: Patch Available (was: Open) > Race condition surrounding in progress snapshot handling in snapshot

[jira] [Updated] (HBASE-21387) Race condition surrounding in progress snapshot handling in snapshot cache leads to loss of snapshot files

2018-11-06 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-21387: --- Attachment: (was: 21387.v9.txt) > Race condition surrounding in progress snapshot handling in snapshot

[jira] [Updated] (HBASE-21387) Race condition surrounding in progress snapshot handling in snapshot cache leads to loss of snapshot files

2018-11-06 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-21387: --- Attachment: 21387.v9.txt > Race condition surrounding in progress snapshot handling in snapshot cache >

[jira] [Updated] (HBASE-21387) Race condition surrounding in progress snapshot handling in snapshot cache leads to loss of snapshot files

2018-11-06 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-21387: --- Status: Open (was: Patch Available) > Race condition surrounding in progress snapshot handling in snapshot

[jira] [Updated] (HBASE-21387) Race condition surrounding in progress snapshot handling in snapshot cache leads to loss of snapshot files

2018-11-06 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-21387: --- Status: Patch Available (was: Open) > Race condition surrounding in progress snapshot handling in snapshot

[jira] [Commented] (HBASE-21387) Race condition surrounding in progress snapshot handling in snapshot cache leads to loss of snapshot files

2018-11-06 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16677347#comment-16677347 ] Ted Yu commented on HBASE-21387: In 21387.v9.txt , I propose another approach. At the beginning of

[jira] [Updated] (HBASE-21387) Race condition surrounding in progress snapshot handling in snapshot cache leads to loss of snapshot files

2018-11-06 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-21387: --- Attachment: 21387.v9.txt > Race condition surrounding in progress snapshot handling in snapshot cache >

[jira] [Updated] (HBASE-21247) Custom Meta WAL Provider doesn't default to custom WAL Provider whose configuration value is outside the enums in Providers

2018-11-06 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-21247: --- Resolution: Fixed Fix Version/s: 2.1.2 2.2.0 Status: Resolved (was:

[jira] [Commented] (HBASE-21347) Backport HBASE-21200 "Memstore flush doesn't finish because of seekToPreviousRow() in memstore scanner." to branch-1

2018-11-06 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16677156#comment-16677156 ] Ted Yu commented on HBASE-21347: Josh: Please go ahead. > Backport HBASE-21200 "Memstore flush doesn't

[jira] [Commented] (SPARK-25954) Upgrade to Kafka 2.1.0

2018-11-06 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16677152#comment-16677152 ] Ted Yu commented on SPARK-25954: Looking at Kafka thread, message from Satish indicated there may be

[jira] [Updated] (HBASE-21247) Custom Meta WAL Provider doesn't default to custom WAL Provider whose configuration value is outside the enums in Providers

2018-11-06 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-21247: --- Attachment: HBASE-21247.branch-2.001.patch > Custom Meta WAL Provider doesn't default to custom WAL

[jira] [Reopened] (HBASE-21247) Custom Meta WAL Provider doesn't default to custom WAL Provider whose configuration value is outside the enums in Providers

2018-11-06 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu reopened HBASE-21247: > Custom Meta WAL Provider doesn't default to custom WAL Provider whose > configuration value is outside the

[jira] [Updated] (HBASE-21247) Custom Meta WAL Provider doesn't default to custom WAL Provider whose configuration value is outside the enums in Providers

2018-11-06 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-21247: --- Attachment: 21247.branch-2.patch > Custom Meta WAL Provider doesn't default to custom WAL Provider whose >

[jira] [Updated] (HBASE-21247) Custom Meta WAL Provider doesn't default to custom WAL Provider whose configuration value is outside the enums in Providers

2018-11-06 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-21247: --- Status: Patch Available (was: Reopened) > Custom Meta WAL Provider doesn't default to custom WAL Provider

[jira] [Reopened] (HBASE-21247) Custom Meta WAL Provider doesn't default to custom WAL Provider whose configuration value is outside the enums in Providers

2018-11-06 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu reopened HBASE-21247: > Custom Meta WAL Provider doesn't default to custom WAL Provider whose > configuration value is outside the

[jira] [Updated] (HBASE-21247) Custom Meta WAL Provider doesn't default to custom WAL Provider whose configuration value is outside the enums in Providers

2018-11-06 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-21247: --- Description: Currently all the WAL Providers acceptable to hbase are specified in Providers enum of

[jira] [Updated] (HBASE-21247) Custom Meta WAL Provider doesn't default to custom WAL Provider whose configuration value is outside the enums in Providers

2018-11-06 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-21247: --- Summary: Custom Meta WAL Provider doesn't default to custom WAL Provider whose configuration value is

[jira] [Commented] (HBASE-21387) Race condition surrounding in progress snapshot handling in snapshot cache leads to loss of snapshot files

2018-11-06 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16676977#comment-16676977 ] Ted Yu commented on HBASE-21387: In patch v9, I added shouldTrackPreviousRound to FileCleanerDelegate,

[jira] [Updated] (HBASE-21387) Race condition surrounding in progress snapshot handling in snapshot cache leads to loss of snapshot files

2018-11-06 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-21387: --- Attachment: two-pass-cleaner.v9.txt > Race condition surrounding in progress snapshot handling in snapshot

[jira] [Updated] (HBASE-21381) Document the hadoop versions using which backup and restore feature works

2018-11-05 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-21381: --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0 Status: Resolved (was:

[jira] [Commented] (HBASE-21347) Backport HBASE-21200 "Memstore flush doesn't finish because of seekToPreviousRow() in memstore scanner." to branch-1

2018-11-05 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16676052#comment-16676052 ] Ted Yu commented on HBASE-21347: lgtm > Backport HBASE-21200 "Memstore flush doesn't finish because of

[jira] [Updated] (HBASE-21387) Race condition surrounding in progress snapshot handling in snapshot cache leads to loss of snapshot files

2018-11-05 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-21387: --- Attachment: two-pass-cleaner.v6.txt > Race condition surrounding in progress snapshot handling in snapshot

[jira] [Updated] (HBASE-21387) Race condition surrounding in progress snapshot handling in snapshot cache leads to loss of snapshot files

2018-11-05 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-21387: --- Attachment: (was: two-pass-cleaner.v6.txt) > Race condition surrounding in progress snapshot handling

[jira] [Comment Edited] (HBASE-21387) Race condition surrounding in progress snapshot handling in snapshot cache leads to loss of snapshot files

2018-11-05 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16675900#comment-16675900 ] Ted Yu edited comment on HBASE-21387 at 11/6/18 12:47 AM: -- In

[jira] [Updated] (HBASE-21387) Race condition surrounding in progress snapshot handling in snapshot cache leads to loss of snapshot files

2018-11-05 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-21387: --- Attachment: two-pass-cleaner.v6.txt > Race condition surrounding in progress snapshot handling in snapshot

[jira] [Updated] (HBASE-21387) Race condition surrounding in progress snapshot handling in snapshot cache leads to loss of snapshot files

2018-11-05 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-21387: --- Attachment: (was: two-pass-cleaner.v5.txt) > Race condition surrounding in progress snapshot handling

[jira] [Commented] (HBASE-21387) Race condition surrounding in progress snapshot handling in snapshot cache leads to loss of snapshot files

2018-11-05 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16675900#comment-16675900 ] Ted Yu commented on HBASE-21387: In two-pass-cleaner.v5.txt , the reference to previous round is changed

[jira] [Updated] (HBASE-21387) Race condition surrounding in progress snapshot handling in snapshot cache leads to loss of snapshot files

2018-11-05 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-21387: --- Attachment: two-pass-cleaner.v5.txt > Race condition surrounding in progress snapshot handling in snapshot

[jira] [Commented] (HBASE-21387) Race condition surrounding in progress snapshot handling in snapshot cache leads to loss of snapshot files

2018-11-05 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16675867#comment-16675867 ] Ted Yu commented on HBASE-21387: bq. holding onto file names in memory We don't need to continue

[jira] [Commented] (HBASE-21387) Race condition surrounding in progress snapshot handling in snapshot cache leads to loss of snapshot files

2018-11-05 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16675805#comment-16675805 ] Ted Yu commented on HBASE-21387: Solving the in progress snapshot race condition is tricky. Please take

[jira] [Updated] (HBASE-21387) Race condition surrounding in progress snapshot handling in snapshot cache leads to loss of snapshot files

2018-11-05 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-21387: --- Attachment: two-pass-cleaner.v4.txt > Race condition surrounding in progress snapshot handling in snapshot

[jira] [Updated] (HBASE-21387) Race condition surrounding in progress snapshot handling in snapshot cache leads to loss of snapshot files

2018-11-05 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-21387: --- Status: Open (was: Patch Available) > Race condition surrounding in progress snapshot handling in snapshot

[jira] [Commented] (HBASE-21381) Document the hadoop versions using which backup and restore feature works

2018-11-05 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16675762#comment-16675762 ] Ted Yu commented on HBASE-21381: liubang: 2.7.x and 2.8.y should be added as supported hadoop releases

[jira] [Commented] (HBASE-21387) Race condition surrounding in progress snapshot handling in snapshot cache leads to loss of snapshot files

2018-11-05 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16675738#comment-16675738 ] Ted Yu commented on HBASE-21387: Thanks for giving the timeline, Josh. The scenario you described is

[jira] [Updated] (HBASE-21438) TestAdmin2#testGetProcedures fails due to FailedProcedure inaccessible

2018-11-05 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-21438: --- Attachment: 21438.v1.txt > TestAdmin2#testGetProcedures fails due to FailedProcedure inaccessible >

[jira] [Commented] (HBASE-21438) TestAdmin2#testGetProcedures fails due to FailedProcedure inaccessible

2018-11-05 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16675573#comment-16675573 ] Ted Yu commented on HBASE-21438: Ran TestAdmin2 with patch which passed. > TestAdmin2#testGetProcedures

[jira] [Updated] (HBASE-21438) TestAdmin2#testGetProcedures fails due to FailedProcedure inaccessible

2018-11-05 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-21438: --- Status: Patch Available (was: Open) > TestAdmin2#testGetProcedures fails due to FailedProcedure

[jira] [Created] (HBASE-21438) TestAdmin2#testGetProcedures fails due to FailedProcedure inaccessible

2018-11-05 Thread Ted Yu (JIRA)
Ted Yu created HBASE-21438: -- Summary: TestAdmin2#testGetProcedures fails due to FailedProcedure inaccessible Key: HBASE-21438 URL: https://issues.apache.org/jira/browse/HBASE-21438 Project: HBase

[jira] [Created] (HBASE-21438) TestAdmin2#testGetProcedures fails due to FailedProcedure inaccessible

2018-11-05 Thread Ted Yu (JIRA)
Ted Yu created HBASE-21438: -- Summary: TestAdmin2#testGetProcedures fails due to FailedProcedure inaccessible Key: HBASE-21438 URL: https://issues.apache.org/jira/browse/HBASE-21438 Project: HBase

[jira] [Commented] (HBASE-21381) Document the hadoop versions using which backup and restore feature works

2018-11-05 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16675530#comment-16675530 ] Ted Yu commented on HBASE-21381: lgtm > Document the hadoop versions using which backup and restore

[jira] [Comment Edited] (HBASE-21246) Introduce WALIdentity interface

2018-11-05 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16675528#comment-16675528 ] Ted Yu edited comment on HBASE-21246 at 11/5/18 6:20 PM: - In patch v24, I

[jira] [Commented] (HBASE-21246) Introduce WALIdentity interface

2018-11-05 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16675528#comment-16675528 ] Ted Yu commented on HBASE-21246: In patch v24, I dropped the static methods from WALFactory - there are

[jira] [Updated] (HBASE-21246) Introduce WALIdentity interface

2018-11-05 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-21246: --- Attachment: 21246.24.txt > Introduce WALIdentity interface > --- > >

[jira] [Commented] (HBASE-21246) Introduce WALIdentity interface

2018-11-05 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16675458#comment-16675458 ] Ted Yu commented on HBASE-21246: replication-src-creates-wal-reader.jpg shows how WAL Reader is created

[jira] [Updated] (HBASE-21246) Introduce WALIdentity interface

2018-11-05 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-21246: --- Attachment: replication-src-creates-wal-reader.jpg > Introduce WALIdentity interface >

[jira] [Updated] (HBASE-21246) Introduce WALIdentity interface

2018-11-05 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-21246: --- Attachment: wal-splitter-writer.jpg wal-splitter-reader.jpg > Introduce WALIdentity

[jira] [Updated] (HBASE-21247) Custom WAL Provider cannot be specified by configuration whose value is outside the enums in Providers

2018-11-05 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-21247: --- Resolution: Fixed Status: Resolved (was: Patch Available) Thanks for the review, Sean and Josh. >

[jira] [Commented] (HBASE-21381) Document the hadoop versions using which backup and restore feature works

2018-11-04 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16674650#comment-16674650 ] Ted Yu commented on HBASE-21381: {code} 61 * 3.2.0+ 62 * 2.9.2+ 63 * 3.0.4+ {code} The

[jira] [Commented] (HBASE-21387) Race condition surrounding in progress snapshot handling in snapshot cache leads to loss of snapshot files

2018-11-04 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16674432#comment-16674432 ] Ted Yu commented on HBASE-21387: Patch v8 adds a boolean, needToCheckInProgressSnapshots, to

[jira] [Updated] (HBASE-21387) Race condition surrounding in progress snapshot handling in snapshot cache leads to loss of snapshot files

2018-11-04 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-21387: --- Attachment: 21387.v8.txt > Race condition surrounding in progress snapshot handling in snapshot cache >

[jira] [Updated] (HBASE-21387) Race condition surrounding in progress snapshot handling in snapshot cache leads to loss of snapshot files

2018-11-02 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-21387: --- Attachment: 21387.v7.txt > Race condition surrounding in progress snapshot handling in snapshot cache >

[jira] [Updated] (HBASE-21387) Race condition surrounding in progress snapshot handling in snapshot cache leads to loss of snapshot files

2018-11-02 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-21387: --- Status: Patch Available (was: Open) > Race condition surrounding in progress snapshot handling in snapshot

[jira] [Commented] (HBASE-21387) Race condition surrounding in progress snapshot handling in snapshot cache leads to loss of snapshot files

2018-11-02 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16673793#comment-16673793 ] Ted Yu commented on HBASE-21387: In patch v6, I try to detect discrepancy w.r.t. the number of in

[jira] [Updated] (HBASE-21387) Race condition surrounding in progress snapshot handling in snapshot cache leads to loss of snapshot files

2018-11-02 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-21387: --- Attachment: 21387.v6.txt > Race condition surrounding in progress snapshot handling in snapshot cache >

[jira] [Commented] (HBASE-21387) Race condition surrounding in progress snapshot handling in snapshot cache leads to loss of snapshot files

2018-11-02 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16673480#comment-16673480 ] Ted Yu commented on HBASE-21387: Currently refreshCache has void return type: {code} private

[jira] [Commented] (HBASE-21387) Race condition surrounding in progress snapshot handling in snapshot cache leads to loss of snapshot files

2018-11-02 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16673467#comment-16673467 ] Ted Yu commented on HBASE-21387: For the unit test, first idea is to use CountDownLatch to reproduce the

[jira] [Updated] (HBASE-21387) Race condition surrounding in progress snapshot handling in snapshot cache leads to loss of snapshot files

2018-11-02 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-21387: --- Attachment: 21387.dbg.txt > Race condition surrounding in progress snapshot handling in snapshot cache >

[jira] [Updated] (HBASE-21387) Race condition surrounding in progress snapshot handling in snapshot cache leads to loss of snapshot files

2018-11-02 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-21387: --- Attachment: (was: 21387.v1.txt) > Race condition surrounding in progress snapshot handling in snapshot

[jira] [Commented] (HBASE-21387) Race condition surrounding in progress snapshot handling in snapshot cache leads to loss of snapshot files

2018-11-02 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16673419#comment-16673419 ] Ted Yu commented on HBASE-21387: Josh, the race condition surrounding in progress snapshot is described

[jira] [Updated] (HBASE-21387) Race condition surrounding in progress snapshot handling in snapshot cache leads to loss of snapshot files

2018-11-02 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-21387: --- Description: During recent report from customer where ExportSnapshot failed: {code} 2018-10-09 18:54:32,559

[jira] [Updated] (HBASE-21387) Race condition surrounding in progress snapshot handling in snapshot cache leads to loss of snapshot files

2018-11-02 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-21387: --- Status: Open (was: Patch Available) > Race condition surrounding in progress snapshot handling in snapshot

[jira] [Commented] (HBASE-21387) Race condition surrounding in progress snapshot handling in snapshot cache leads to loss of snapshot files

2018-11-02 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16673171#comment-16673171 ] Ted Yu commented on HBASE-21387: >From https://builds.apache.org/job/PreCommit-HBASE-Build/14932/console

[jira] [Commented] (HBASE-21387) Race condition surrounding in progress snapshot handling in snapshot cache leads to loss of snapshot files

2018-11-01 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16672436#comment-16672436 ] Ted Yu commented on HBASE-21387: {code} [ERROR]

[jira] [Comment Edited] (HBASE-21246) Introduce WALIdentity interface

2018-11-01 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16670472#comment-16670472 ] Ted Yu edited comment on HBASE-21246 at 11/2/18 12:27 AM: -- wal-providers.png is

[jira] [Comment Edited] (HBASE-21246) Introduce WALIdentity interface

2018-11-01 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16670472#comment-16670472 ] Ted Yu edited comment on HBASE-21246 at 11/2/18 12:04 AM: -- wal-providers.png is

[jira] [Comment Edited] (HBASE-21246) Introduce WALIdentity interface

2018-11-01 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16670472#comment-16670472 ] Ted Yu edited comment on HBASE-21246 at 11/1/18 11:58 PM: -- wal-providers.png is

[jira] [Commented] (HBASE-21387) Race condition surrounding in progress snapshot handling in snapshot cache leads to loss of snapshot files

2018-11-01 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16672271#comment-16672271 ] Ted Yu commented on HBASE-21387: You're right - with the Filter in place, the check is not needed. >

[jira] [Updated] (HBASE-21387) Race condition surrounding in progress snapshot handling in snapshot cache leads to loss of snapshot files

2018-11-01 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-21387: --- Attachment: 21387.v3.txt > Race condition surrounding in progress snapshot handling in snapshot cache >

[jira] [Commented] (HBASE-21387) Race condition surrounding in progress snapshot handling in snapshot cache leads to loss of snapshot files

2018-11-01 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16672246#comment-16672246 ] Ted Yu commented on HBASE-21387: Current code would include in progress snapshot(s): {code}

[jira] [Updated] (HBASE-21387) Race condition surrounding in progress snapshot handling in snapshot cache leads to loss of snapshot files

2018-11-01 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-21387: --- Summary: Race condition surrounding in progress snapshot handling in snapshot cache leads to loss of

[jira] [Updated] (HBASE-21387) Race condition in snapshot cache refreshing leads to loss of snapshot files

2018-11-01 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-21387: --- Attachment: 21387.v2.txt > Race condition in snapshot cache refreshing leads to loss of snapshot files >

[jira] [Updated] (HBASE-21387) Race condition in snapshot cache refreshing leads to loss of snapshot files

2018-11-01 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-21387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-21387: --- Description: During recent report from customer where ExportSnapshot failed: {code} 2018-10-09 18:54:32,559

<    1   2   3   4   5   6   7   8   9   10   >