Dear Wiki user, You have subscribed to a wiki page or wiki category on "Hadoop Wiki" for change notification.
The "Release-2.6.1-Working-Notes" page has been changed by VinodKumarVavilapalli: https://wiki.apache.org/hadoop/Release-2.6.1-Working-Notes?action=diff&rev1=16&rev2=17 Comment: Updating progress from others on local cherry-picks Ordered list of commits to cherrypick ||SHA1||JIRA || Status || New patch on JIRA || Applies cleanly || - ||5f3d967aaefa0b20ef1586b4048b8fa5345d2618 ||HDFS-7278. Add a command that allows sysadmins to manually trigger full block reports from || || || ADDED!!! minor issues || + ||5f3d967aaefa0b20ef1586b4048b8fa5345d2618 ||HDFS-7278. Add a command that allows sysadmins to manually trigger full block reports from || || || ADDED as a dependency. Minor issues. TODO: Feature || ||946463efefec9031cacb21d5a5367acd150ef904 ||HDFS-7213. processIncrementalBlockReport performance degradation. Contributed by Eric Payn || || || yes || ||842a54a5f66e76eb79321b66cc3b8820fe66c5cd ||HDFS-7235. DataNode#transferBlock should report blocks that don't exist using reportBadBlo || || || yes || ||8bfef590295372a48bd447b1462048008810ee17 ||HDFS-7263. Snapshot read can reveal future bytes for appended files. Contributed by Tao Lu || || || yes || @@ -20, +20 @@ ||f6d1bf5ed1cf647d82e676df15587de42b1faa42 ||HADOOP-11333. Fix deadlock in DomainSocketWatcher when the notification pipe is full (zhao || || ||remove; already committed || ||38ea1419f60d2b8176dba4931748f1f0e52ca84e ||YARN-2905. AggregatedLogsBlock page can infinitely loop if the aggregated log file is corr ||Applied locally || ||Yes || ||d21ef79707a0f32939d9a5af4fed2d9f5fe6f2ec ||YARN-2890. MiniYARNCluster should start the timeline server based on the configuration. Co ||Applied locally || ||Yes || - ||06552a15d5172a2b0ad3d61aa7f9a849857385aa ||HDFS-7446. HDFS inotify should have the ability to determine what txid it has read up to ( || || || minor issues || + ||06552a15d5172a2b0ad3d61aa7f9a849857385aa ||HDFS-7446. HDFS inotify should have the ability to determine what txid it has read up to ( || || || minor issues. TODO: Is this incompatible || ||d6f3d4893d750f19dd8c539fe28eecfab2a54576 ||YARN-2894. Fixed a bug regarding application view acl when RM fails over. Contributed by R ||Applied locally || || No, minor import issues || ||25be97808b99148412c0efd4d87fc750db4d6607 ||YARN-2874. Dead lock in DelegationTokenRenewer which blocks RM to execute any further apps ||Applied locally || ||Yes || ||dabdd2d746d1e1194c124c5c7fe73fcc025e78d2 ||HADOOP-11343. Overflow is not properly handled in caclulating final iv for AES CTR. Contri ||Applied locally || ||Yes || @@ -45, +45 @@ ||75e4e55e12b2faa521af7c23fddcba06a9ce661d ||HDFS-7596. NameNode should prune dead storages from storageMap. Contributed by Arpit Agarw || || || yes || ||33534a0c9aef5024aa6f340e7ee24930c8fa8ed5 ||HDFS-7533. Datanode sometimes does not shutdown on receiving upgrade shutdown command. Con || || || yes || ||2e4df8710435c8362506fe944a935e74ad5919c0 ||HADOOP-11350. The size of header buffer of HttpServer is too small when HTTPS is enabled. ||Applied locally || ||Yes || - ||7779f38e68ca4e0f7ac08eb7e5f4801b89979d02 ||HDFS-7610. Fix removal of dynamically added DN volumes (Lei (Eddy) Xu via Colin P. McCabe) || || || ADDED!! minor issues || + ||7779f38e68ca4e0f7ac08eb7e5f4801b89979d02 ||HDFS-7610. Fix removal of dynamically added DN volumes (Lei (Eddy) Xu via Colin P. McCabe) || || || ADDED as a dependency. minor issues || ||1d9d166c0beb56aa45e65f779044905acff25d88 ||HDFS-7575. Upgrade should generate a unique storage ID for each volume. (Contributed by Ar || || || yes || ||7b69719455a1a374c9417417ef0c8d7ba6bf593f ||HADOOP-11482. Use correct UGI when KMSClientProvider is called by a proxy user. Contribute ||Applied locally || ||Yes || - ||24f0d56afb853a371f905c4569d82d09d89cf13e ||HDFS-7676. Fix TestFileTruncate to avoid bug of HDFS-7611. Contributed by Konstantin Shvac || || || remove; does not apply to 2.6 || + ||24f0d56afb853a371f905c4569d82d09d89cf13e ||HDFS-7676. Fix TestFileTruncate to avoid bug of HDFS-7611. Contributed by Konstantin Shvac || || || remove; does not apply to 2.6, as we are dropping HDFS-7611 itself || ||8100c8a68c32978a177af9a3e6639f6de533886d ||YARN-3011. Possible IllegalArgumentException in ResourceLocalizationService might lead NM ||Applied locally || ||No, merge conflict in ResourceLocalizationService || ||12522fd9cbd8da8c040a5b7bb71fcdaa256daf89 ||YARN-3103. AMRMClientImpl does not update AMRM token properly. Contributed by Jason Lowe ( ||Applied locally || ||Yes || - ||e8300957a75353849219d616cfac08b08c182db2 ||HDFS-7611. deleteSnapshot and delete of a file can leave orphaned blocks in the blocksMap || || || remove; dropped from 2.6.1 || + ||e8300957a75353849219d616cfac08b08c182db2 ||HDFS-7611. deleteSnapshot and delete of a file can leave orphaned blocks in the blocksMap || || || remove; dropped from 2.6.1. Highly problematic patch. It depends on HDFS-7573, which in turn depends on HDFS-7438 and possibly others. The latter 2 JIRAs are part of HDFS-7416, a much larger effort. Workaround: Leave safe mode manually and Delete the orphaned files. || ||43b3b43cea1f620ce66521bcc1c4b6aec264aa9a ||MAPREDUCE-6230. Fixed RMContainerAllocator to update the new AMRMToken service name proper ||Applied locally || ||Yes || ||c428d303f67bef3a7df12153947c6b0199a0938b ||HDFS-7707. Edit log corruption due to delayed block removal again. Contributed by Yongjun || || || yes || ||5807afed0a4f08b6b2acd88f424bace506e03707 ||HDFS-7733. NFS: readdir/readdirplus return null directory attribute on failure. (Contribut || || || remove; already committed || @@ -66, +66 @@ ||187e081d5a8afe1ddfe5d7b5e7de7a94512aa53e ||HADOOP-11604. Prevent ConcurrentModificationException while closing domain sockets during ||Applied locally || ||Yes || ||fefeba4ac8bed44ce2dd0d3c4f0a99953ff8d4df ||YARN-3238. Connection timeouts to nodemanagers are retried at multiple levels. Contributed ||Applied locally || ||Yes || ||657a6e389b3f6eae43efb11deb6253c3b1255a51 ||HDFS-7009. Active NN and standby NN have different live nodes. Contributed by Ming Ma. (c || || || minor issues || - ||8346427929d2b6f2fd3fa228d77f3cf596ef0306 ||HDFS-7831. Fix the starting index and end condition of the loop in FileDiffList.findEarlie || || || remove; does not apply to 2.6 || + ||8346427929d2b6f2fd3fa228d77f3cf596ef0306 ||HDFS-7831. Fix the starting index and end condition of the loop in FileDiffList.findEarlie || || || remove; does not apply to 2.6, vinodkv: Sangjin, why? || ||efb7e287f45c6502f293456034a37d9209a917be ||HDFS-7763. fix zkfc hung issue due to not catching exception in a corner case. Contributed || || || yes || ||257087417e424e628f090b6b648ccb3b9c880250 ||YARN-3239. WebAppProxy does not support a final tracking url which has query fragments and ||Applied locally || ||No, minor merge conflicts in WebAppProxyServlet.java || - ||12fe3afcd3ea224909bd125f5c6a7410bf2601b0 ||HDFS-7843. A truncated file is corrupted after rollback from a rolling upgrade. || || || remove; does not apply to 2.6 || + ||12fe3afcd3ea224909bd125f5c6a7410bf2601b0 ||HDFS-7843. A truncated file is corrupted after rollback from a rolling upgrade. || || || remove; does not apply to 2.6, vinodkv: Sangjin, why? || ||6090f51725e2b44d794433ed72a1901fae2ba7e3 ||HDFS-7871. NameNodeEditLogRoller can keep printing 'Swallowing exception' message. Contrib || || || yes || ||888a44563819ba910dc3cc10d10ee0fb8f05db61 ||YARN-3222. Fixed NPE on RMNodeImpl#ReconnectNodeTransition when a node is reconnected with ||Applied locally || ||Yes || ||721d7b574126c4070322f70ec5b49a7b8558a4c7 ||YARN-3231. FairScheduler: Changing queueMaxRunningApps interferes with pending jobs. (Siqi ||Applied locally || ||Yes || @@ -82, +82 @@ ||eefca23e8c5e474de1e25bf2ec8a5b266bbe8cfe ||HDFS-7830. DataNode does not release the volume lock when adding a volume fails. (Lei Xu v || || || non-trivial changes || ||813c93cb250d6d556604fe98845b979970bd5e18 ||HADOOP-11710. Make CryptoOutputStream behave like DFSOutputStream wrt synchronization. (Se ||Applied locally || ||Yes || ||44aedad5ddc8069a6dba3eaf66ed54d612b21208 ||YARN-3267. Timelineserver applies the ACL rules after applying the limit on the number of || || - ||af80a98ace50934284dde417efc802bb094c8b4e ||HDFS-7926. NameNode implementation of ClientProtocol.truncate(..) is not idempotent. Contr || || || remove; does not apply to 2.6 || + ||af80a98ace50934284dde417efc802bb094c8b4e ||HDFS-7926. NameNode implementation of ClientProtocol.truncate(..) is not idempotent. Contr || || || remove; does not apply to 2.6, truncate is a 2.7 feature || ||5a5b2446485531f12d37f3d4ca791672b9921872 ||HDFS-7587. Edit log corruption can happen if append fails with a quota violation. Contribu || || yes || used the 2.6 patch || ||219eb22c1571f76df32967a930049d983cbf5024 ||HDFS-7929. inotify unable fetch pre-upgrade edit log segments once upgrade starts (Zhe Zha || || || non-trivial changes || ||90164ffd84f6ef56e9f8f99dcc7424a8d115dbae ||HDFS-7930. commitBlockSynchronization() does not remove locations. (yliu) || || yes || used the 2.6 patch || @@ -103, +103 @@ ||756c2542930756fef1cbff82056b418070f8d55f ||MAPREDUCE-6238. MR2 can't run local jobs with -libjars command options which is a regressi || || ||1544c63602089b690e850e0e30af4589513a2371 ||HADOOP-11812. Implement listLocatedStatus for ViewFileSystem to speed up split calculation ||Applied locally || ||Yes || ||a6a5d1d6b5ee76c829ba7b54a4ad619f7b986681 ||HADOOP-11730. Regression: s3n read failure recovery broken. (Takenori Sato via stevel) ||Applied locally || ||Yes || - ||788b76761d5dfadf688406d50169e95401fe5d33 ||HADOOP-11802. DomainSocketWatcher thread terminates sometimes after there is an I/O error ||Skipped ||Yes ||No, need to commit HDFS-7915 first. If we commit HDFS-7915, we need to commit HDFS-8070 as well. || + ||788b76761d5dfadf688406d50169e95401fe5d33 ||HADOOP-11802. DomainSocketWatcher thread terminates sometimes after there is an I/O error ||Skipped ||Yes ||Dropped. We will need to commit HDFS-7915 first. If we commit HDFS-7915, we need to commit HDFS-8070 as well. || ||4045c41afe440b773d006e962bf8a5eae3fdc284 ||YARN-3464. Race condition in LocalizerRunner kills localizer before localizing all resourc || || ||32dc13d907a416049bdb7deff429725bd6dbcb49 ||MAPREDUCE-6324. Fixed MapReduce uber jobs to not fail the udpate of AM-RM tokens when they || || ||58970d69de8a1662e4548cd6d4ca460dd70562f8 ||HADOOP-11491. HarFs incorrectly declared as requiring an authority. (Brahma Reddy Battula ||Applied locally || ||Yes || @@ -120, +120 @@ ||d3193fd1d7395bf3e7c8dfa70d1aec08b0f147e6 ||Move YARN-2918 from 2.8.0 to 2.7.1 (cherry picked from commit 03f897fd1a3779251023bae3582 || || ||839f81a6326b2f8b3d5183178382c1551b0bc259 ||YARN-3700. Made generic history service load a number of latest applications according to || || ||25db34127811fbadb9a698fa3a76e24d426fb0f6 ||HDFS-8431. hdfs crypto class not found in Windows. Contributed by Anu Engineer. (cherry p || || || minor issues || - ||33648268ce0f79bf51facafa3d151612e3d00ddb ||HADOOP-11934. Use of JavaKeyStoreProvider in LdapGroupsMapping causes infinite loop. Contr ||Skipped || ||No, cannot apply to branch-2.6 due to java.nio package. || + ||33648268ce0f79bf51facafa3d151612e3d00ddb ||HADOOP-11934. Use of JavaKeyStoreProvider in LdapGroupsMapping causes infinite loop. Contr ||Skipped || ||Dropped. Cannot apply to branch-2.6 due to java.nio package. || ||17fb442a4c4e43105374c97fccd68dd966729a19 ||HDFS-7609. Avoid retry cache collision when Standby NameNode loading edits. Contributed by || || || non-trivial changes || ||4fee8b320276bac86278e1ae0a3397592a78aa18 ||YARN-2900. Application (Attempt and Container) Not Found in AHS results in Internal Server || || ||a3734f67d35e714690ecdf21d80bce8a355381e3 ||YARN-3725. App submission via REST API is broken in secure mode due to Timeline DT service || || @@ -136, +136 @@ ||0221d19f4e398c386f4ca3990b0893562aa8dacf ||YARN-3850. NM fails to read files from full disks which can lead to container logs being l || || ||516bbf1c20547dc513126df0d9f0934bb65c10c7 ||HDFS-7314. When the DFSClient lease cannot be renewed, abort open-for-write files rather t || || || minor issues || ||c31e3ba92132f232bd56b257f3854ffe430fbab9 ||YARN-3990. AsyncDispatcher may overloaded with RMAppNodeUpdateEvent when Node is connected || || - ||d2b941f94a835f7bdde7714d21a470b505aa582b ||HDFS-8850. VolumeScanner thread exits with exception if there is no block pool to be scann || || || remove; does not apply to 2.6 || + ||d2b941f94a835f7bdde7714d21a470b505aa582b ||HDFS-8850. VolumeScanner thread exits with exception if there is no block pool to be scann || || || remove; does not apply to 2.6. vinodkv: Sangjin, why? || ||5950c1f6f8ac6f514f8d2e8bfbd1f71747b097de ||HADOOP-11932. MetricsSinkAdapter may hang when being stopped. Contributed by Brahma Reddy ||Applied locally ||Yes ||No, conflict in TestMetricsSystemImpl. ||
