Re: data node reads blocks sequentially from disk

2020-03-11 Thread Todd Lipcon
We also issue explicit readahead via fadvise since 2011 or so, so the typical io sizes hitting the device are large enough to max out the throughput, at least for typical spinning disks. Todd On Wed, Mar 11, 2020, 9:46 AM Kihwal Lee wrote: > When Datanode was initially designed, Linux AIO was

[jira] [Resolved] (HDFS-14535) The default 8KB buffer in requestFileDescriptors#BufferedOutputStream is causing lots of heap allocation in HBase when using short-circut read

2019-06-04 Thread Todd Lipcon (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon resolved HDFS-14535. Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.3.0 > The default 8KB buf

[jira] [Resolved] (HDFS-14482) Crash when using libhdfs with bad classpath

2019-05-14 Thread Todd Lipcon (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon resolved HDFS-14482. Resolution: Fixed > Crash when using libhdfs with bad classp

[jira] [Created] (HDFS-14482) Crash when using libhdfs with bad classpath

2019-05-08 Thread Todd Lipcon (JIRA)
Todd Lipcon created HDFS-14482: -- Summary: Crash when using libhdfs with bad classpath Key: HDFS-14482 URL: https://issues.apache.org/jira/browse/HDFS-14482 Project: Hadoop HDFS Issue Type: Bug

[jira] [Created] (HDFS-14111) hdfsOpenFile on HDFS causes unnecessary IO from file offset 0

2018-11-28 Thread Todd Lipcon (JIRA)
Todd Lipcon created HDFS-14111: -- Summary: hdfsOpenFile on HDFS causes unnecessary IO from file offset 0 Key: HDFS-14111 URL: https://issues.apache.org/jira/browse/HDFS-14111 Project: Hadoop HDFS

[jira] [Resolved] (HDFS-10369) hdfsread crash when reading data reaches to 128M

2018-11-28 Thread Todd Lipcon (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon resolved HDFS-10369. Resolution: Invalid You're mallocing a buffer of 5 bytes here, seems your C code is just broken

Re: [DISCUSS] Hadoop RPC encryption performance improvements

2018-11-02 Thread Todd Lipcon
using SSL). But it looks like both attempts have not been > progressing. > > > > > > > > During the recent Hadoop contributor meetup, Daryn Sharp mentioned > he's > > > > working on another approach that leverages Netty for its SSL > > encryption, &

[jira] [Created] (HDFS-13826) Add a hidden configuration for NameNode to generate fake block locations

2018-08-14 Thread Todd Lipcon (JIRA)
Todd Lipcon created HDFS-13826: -- Summary: Add a hidden configuration for NameNode to generate fake block locations Key: HDFS-13826 URL: https://issues.apache.org/jira/browse/HDFS-13826 Project: Hadoop

[jira] [Created] (HDFS-13747) Statistic for list_located_status is incremented incorrectly by listStatusIterator

2018-07-18 Thread Todd Lipcon (JIRA)
Todd Lipcon created HDFS-13747: -- Summary: Statistic for list_located_status is incremented incorrectly by listStatusIterator Key: HDFS-13747 URL: https://issues.apache.org/jira/browse/HDFS-13747 Project

Re: [DISCUSS]: securing ASF Hadoop releases out of the box

2018-07-05 Thread Todd Lipcon
gt; sophisticated > > multinode deployments, should the scripts refuse to work without > kerberos > > unless you pass in some argument like "--Dinsecure-clusters- > permitted" > > > > Any other ideas? > > > > > > > - > > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org > > For additional commands, e-mail: common-dev-h...@hadoop.apache.org > > > > > > > -- Todd Lipcon Software Engineer, Cloudera

[jira] [Created] (HDFS-13703) Avoid allocation of CorruptedBlocks hashmap when no corrupted blocks are hit

2018-06-26 Thread Todd Lipcon (JIRA)
Todd Lipcon created HDFS-13703: -- Summary: Avoid allocation of CorruptedBlocks hashmap when no corrupted blocks are hit Key: HDFS-13703 URL: https://issues.apache.org/jira/browse/HDFS-13703 Project

[jira] [Created] (HDFS-13702) HTrace hooks taking 10-15% CPU in DFS client when disabled

2018-06-26 Thread Todd Lipcon (JIRA)
Todd Lipcon created HDFS-13702: -- Summary: HTrace hooks taking 10-15% CPU in DFS client when disabled Key: HDFS-13702 URL: https://issues.apache.org/jira/browse/HDFS-13702 Project: Hadoop HDFS

[jira] [Created] (HDFS-13701) Removal of logging guards regressed performance

2018-06-26 Thread Todd Lipcon (JIRA)
Todd Lipcon created HDFS-13701: -- Summary: Removal of logging guards regressed performance Key: HDFS-13701 URL: https://issues.apache.org/jira/browse/HDFS-13701 Project: Hadoop HDFS Issue Type

[jira] [Resolved] (HDFS-3653) 1.x: Add a retention period for purged edit logs

2018-04-23 Thread Todd Lipcon (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon resolved HDFS-3653. --- Resolution: Won't Fix > 1.x: Add a retention period for purged edit l

[jira] [Resolved] (HDFS-3069) If an edits file has more edits in it than expected by its name, should trigger an error

2018-04-23 Thread Todd Lipcon (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon resolved HDFS-3069. --- Resolution: Won't Fix Target Version/s: (was: ) > If an edits file has more ed

Re: Datanode synchronization is horrible. I’m thinking we can use ReentrantReadWriteLock for synchronization. What do you guys think?

2015-02-17 Thread Todd Lipcon
thinking we can use ReentrantReadWriteLock for synchronization. What do you guys think? -- Todd Lipcon Software Engineer, Cloudera

[jira] [Resolved] (HDFS-957) FSImage layout version should be only once file is complete

2015-02-11 Thread Todd Lipcon (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon resolved HDFS-957. -- Resolution: Won't Fix FSImage layout version should be only once file is complete

[jira] [Resolved] (HDFS-3528) Use native CRC32 in DFS write path

2014-08-28 Thread Todd Lipcon (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon resolved HDFS-3528. --- Resolution: Fixed Fix Version/s: 2.6.0 Use native CRC32 in DFS write path

Re: Updates on migration to git

2014-08-26 Thread Todd Lipcon
system. Thank You. -- Todd Lipcon Software Engineer, Cloudera

[jira] [Resolved] (HDFS-3278) Umbrella Jira for HDFS-HA Phase 2

2014-05-16 Thread Todd Lipcon (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon resolved HDFS-3278. --- Resolution: Fixed Fix Version/s: 2.1.0-beta Assignee: Todd Lipcon (was: Sanjay Radia

Re: confirm expected HA behavior

2014-02-07 Thread Todd Lipcon
that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- Todd Lipcon Software Engineer

Re: [VOTE] Merge HDFS-5698 Use protobuf to serialize / deserialize FSImage to trunk

2014-01-30 Thread Todd Lipcon
covered this feature. Once the feature is merged into trunk, we will continue to test and fix any bugs that may be found on trunk. The bulk of the design and implementation was done by Jing Zhao, Suresh Srinivas and me. Thanks Kihwal Lee, Todd Lipcon, Colin Patrick McCabe, Chris Nauroth, and Daryn

[jira] [Created] (HDFS-5790) LeaseManager.findPath is very slow when many leases need recovery

2014-01-16 Thread Todd Lipcon (JIRA)
Todd Lipcon created HDFS-5790: - Summary: LeaseManager.findPath is very slow when many leases need recovery Key: HDFS-5790 URL: https://issues.apache.org/jira/browse/HDFS-5790 Project: Hadoop HDFS

Re: Deprecate BackupNode

2013-12-04 Thread Todd Lipcon
You. -- Todd Lipcon Software Engineer, Cloudera

Re: Next releases

2013-11-12 Thread Todd Lipcon
On Mon, Nov 11, 2013 at 2:57 PM, Colin McCabe cmcc...@alumni.cmu.eduwrote: To be honest, I'm not aware of anything in 2.2.1 that shouldn't be there. However, I have only been following the HDFS and common side of things so I may not have the full picture. Arun, can you give a specific

Re: Question regarding access to different hadoop 2.0 cluster

2013-11-06 Thread Todd Lipcon
of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- Todd Lipcon Software Engineer, Cloudera

BKJM - anyone using it?

2013-11-05 Thread Todd Lipcon
to find out if it is indeed abandoned) -Todd -- Todd Lipcon Software Engineer, Cloudera

Re: Datanode fencing mechanism

2013-10-28 Thread Todd Lipcon
the bloc I am use the CDH4.3.1 version, and am reading the DataNode code. I don't find the DataNode to save the promise as part of persistent state. I want to know whether the case 3 is handled in CDH4.3.1 version. If the case is hadnled, where is the code? Thanks, LiuLe -- Todd

[jira] [Created] (HDFS-5287) JN need not validate finalized log segments in newEpoch

2013-10-01 Thread Todd Lipcon (JIRA)
Todd Lipcon created HDFS-5287: - Summary: JN need not validate finalized log segments in newEpoch Key: HDFS-5287 URL: https://issues.apache.org/jira/browse/HDFS-5287 Project: Hadoop HDFS Issue

Re: Long time fail over when using QJM

2013-08-29 Thread Todd Lipcon
there's no log in 12 minutes. There is no long gc happened. It seems the code blocked somewhere. Unfortunately, I forgot to print the jstack info T_T. Hope for your response. Best regards, Mickey -- Todd Lipcon Software Engineer, Cloudera

Re: Secure deletion of blocks

2013-08-15 Thread Todd Lipcon
Software Ltd Registered No. 06340261 Signal House, Jacklyns Lane, Alresford, Hampshire, SO24 9JJ, U.K. __**__ -- Todd Lipcon Software Engineer, Cloudera

[jira] [Resolved] (HDFS-3656) ZKFC may write a null breadcrumb znode

2013-08-14 Thread Todd Lipcon (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon resolved HDFS-3656. --- Resolution: Duplicate Target Version/s: (was: ) Yep, I think you're right. Thanks

[jira] [Created] (HDFS-5074) Allow starting up from an fsimage checkpoint in the middle of a segment

2013-08-06 Thread Todd Lipcon (JIRA)
Todd Lipcon created HDFS-5074: - Summary: Allow starting up from an fsimage checkpoint in the middle of a segment Key: HDFS-5074 URL: https://issues.apache.org/jira/browse/HDFS-5074 Project: Hadoop HDFS

[jira] [Created] (HDFS-5058) QJM should validate startLogSegment() more strictly

2013-08-02 Thread Todd Lipcon (JIRA)
Todd Lipcon created HDFS-5058: - Summary: QJM should validate startLogSegment() more strictly Key: HDFS-5058 URL: https://issues.apache.org/jira/browse/HDFS-5058 Project: Hadoop HDFS Issue Type

[jira] [Created] (HDFS-5037) Active NN should trigger its own edit log rolls

2013-07-26 Thread Todd Lipcon (JIRA)
Todd Lipcon created HDFS-5037: - Summary: Active NN should trigger its own edit log rolls Key: HDFS-5037 URL: https://issues.apache.org/jira/browse/HDFS-5037 Project: Hadoop HDFS Issue Type

Re: I'm interested in working with HDFS-4680. Can somebody be a mentor?

2013-07-17 Thread Todd Lipcon
/mentoringprogramme-icfoss-pilot.html [5] https://issues.apache.org/jira/browse/HBASE-6449 Thank you, Sreejith R -- Todd Lipcon Software Engineer, Cloudera

[jira] [Created] (HDFS-4982) JournalNode should relogin from keytab before fetching logs from other JNs

2013-07-11 Thread Todd Lipcon (JIRA)
Todd Lipcon created HDFS-4982: - Summary: JournalNode should relogin from keytab before fetching logs from other JNs Key: HDFS-4982 URL: https://issues.apache.org/jira/browse/HDFS-4982 Project: Hadoop

Re: A question for txid

2013-06-25 Thread Todd Lipcon
{ editLog.setNextTxId(lastAppliedTxId + 1L); } Is it possible that (lastAppliedTxId + 1L) exceed Long.MAX_VALUE ? -- Harsh J -- Todd Lipcon Software Engineer, Cloudera

Re: DesignLounge @ HadoopSummit

2013-06-24 Thread Todd Lipcon
there. Look forward to seeing you all at the summit, E14 PS Please forward to the other -dev lists. This event is for folks on the -dev lists. -- http://hortonworks.com/download/ -- Todd Lipcon Software Engineer, Cloudera

Re: DesignLounge @ HadoopSummit

2013-06-24 Thread Todd Lipcon
with this format is. E14 On Jun 24, 2013, at 2:50 PM, Todd Lipcon t...@cloudera.com wrote: Hey all, Unfortunately I will have to leave early on Thursday, so will miss the slotted time at the design lounge. But, I'll be at the summit all day Wed and on Thurs morning, so if anyone wants to catch

Re: Heads up: branch-2.1-beta

2013-06-20 Thread Todd Lipcon
. At this point I'm actually reasonably concerned and will follow up tomorrow. Thanks, Roman. -- Todd Lipcon Software Engineer, Cloudera

[jira] [Created] (HDFS-4915) Add config to ZKFC to disable fencing

2013-06-18 Thread Todd Lipcon (JIRA)
Todd Lipcon created HDFS-4915: - Summary: Add config to ZKFC to disable fencing Key: HDFS-4915 URL: https://issues.apache.org/jira/browse/HDFS-4915 Project: Hadoop HDFS Issue Type: Bug

Re: dfs.datanode.socket.reuse.keepalive

2013-06-10 Thread Todd Lipcon
this will inevitably lead to clients trying to use sockets which have gone stale, because the server closes them way before the client expires them. Unless I'm missing something, we should probably either lengthen the keepalive, or shorten the socket cache expiry, or both. thoughts? Colin -- Todd

[jira] [Created] (HDFS-4879) Add blocked ArrayList collection to avoid CMS full GCs

2013-06-04 Thread Todd Lipcon (JIRA)
Todd Lipcon created HDFS-4879: - Summary: Add blocked ArrayList collection to avoid CMS full GCs Key: HDFS-4879 URL: https://issues.apache.org/jira/browse/HDFS-4879 Project: Hadoop HDFS Issue

[jira] [Resolved] (HDFS-4184) Add the ability for Client to provide more hint information for DataNode to manage the OS buffer cache more accurate

2013-05-28 Thread Todd Lipcon (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon resolved HDFS-4184. --- Resolution: Duplicate Add the ability for Client to provide more hint information for DataNode

[jira] [Created] (HDFS-4828) Make QJM epoch-related errors more understandable

2013-05-16 Thread Todd Lipcon (JIRA)
Todd Lipcon created HDFS-4828: - Summary: Make QJM epoch-related errors more understandable Key: HDFS-4828 URL: https://issues.apache.org/jira/browse/HDFS-4828 Project: Hadoop HDFS Issue Type

[jira] [Created] (HDFS-4833) Corrupt blocks are not invalidated when first processing repl queues

2013-05-16 Thread Todd Lipcon (JIRA)
Todd Lipcon created HDFS-4833: - Summary: Corrupt blocks are not invalidated when first processing repl queues Key: HDFS-4833 URL: https://issues.apache.org/jira/browse/HDFS-4833 Project: Hadoop HDFS

[jira] [Created] (HDFS-4799) Corrupt replica can be prematurely removed from corruptReplicas map

2013-05-05 Thread Todd Lipcon (JIRA)
Todd Lipcon created HDFS-4799: - Summary: Corrupt replica can be prematurely removed from corruptReplicas map Key: HDFS-4799 URL: https://issues.apache.org/jira/browse/HDFS-4799 Project: Hadoop HDFS

Re: DistributedFileSystem.listStatus() - Why does it do partial listings then assemble?

2013-05-02 Thread Todd Lipcon
/DistributedFileSystem.java http://docs.oracle.com/javase/6/docs/api/java/util/Iterator.html thanks! -bc -- Todd Lipcon Software Engineer, Cloudera

Re: Heads up - Snapshots feature merge into trunk

2013-04-24 Thread Todd Lipcon
in HdfsAdmin, or alternatively in DistributedFileSystem, forcing a user to down-cast if they want to use the HDFS-specific operation. -Todd -- Todd Lipcon Software Engineer, Cloudera

Re: VOTE: HDFS-347 merge

2013-04-08 Thread Todd Lipcon
the insecure SCR code until secure SCR can be implemented for Windows. Please cast your vote by EOD Monday 4/8. best, Colin -- Todd Lipcon Software Engineer, Cloudera

Re: VOTE: HDFS-347 merge

2013-04-04 Thread Todd Lipcon
. Please cast your vote by EOD Monday 4/8. best, Colin -- Todd Lipcon Software Engineer, Cloudera

[jira] [Created] (HDFS-4643) Fix flakiness in TestQuorumJournalManager

2013-03-27 Thread Todd Lipcon (JIRA)
Todd Lipcon created HDFS-4643: - Summary: Fix flakiness in TestQuorumJournalManager Key: HDFS-4643 URL: https://issues.apache.org/jira/browse/HDFS-4643 Project: Hadoop HDFS Issue Type: Bug

[jira] [Resolved] (HDFS-4538) allow use of legacy blockreader

2013-03-27 Thread Todd Lipcon (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon resolved HDFS-4538. --- Resolution: Fixed Hadoop Flags: Reviewed Committed to HDFS-347 branch. allow

[jira] [Resolved] (HDFS-4617) warning while purging logs with QJM enabled

2013-03-27 Thread Todd Lipcon (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon resolved HDFS-4617. --- Resolution: Duplicate ATM points out that I already found this bug 3 months ago... resolving

[jira] [Created] (HDFS-4618) default for checkpoint txn interval is too low

2013-03-20 Thread Todd Lipcon (JIRA)
Todd Lipcon created HDFS-4618: - Summary: default for checkpoint txn interval is too low Key: HDFS-4618 URL: https://issues.apache.org/jira/browse/HDFS-4618 Project: Hadoop HDFS Issue Type: Bug

[jira] [Created] (HDFS-4621) additional logging to help diagnose slow QJM logSync

2013-03-20 Thread Todd Lipcon (JIRA)
Todd Lipcon created HDFS-4621: - Summary: additional logging to help diagnose slow QJM logSync Key: HDFS-4621 URL: https://issues.apache.org/jira/browse/HDFS-4621 Project: Hadoop HDFS Issue Type

[jira] [Created] (HDFS-4617) warning while purging logs with QJM enabled

2013-03-19 Thread Todd Lipcon (JIRA)
Todd Lipcon created HDFS-4617: - Summary: warning while purging logs with QJM enabled Key: HDFS-4617 URL: https://issues.apache.org/jira/browse/HDFS-4617 Project: Hadoop HDFS Issue Type: Bug

Re: [Vote] Merge branch-trunk-win to trunk

2013-02-27 Thread Todd Lipcon
that with the existing commit process. Let me know if you have any questions. Regards, Suresh -- http://hortonworks.com/download/ -- http://hortonworks.com/download/ -- Todd Lipcon Software Engineer, Cloudera

Re: VOTE: HDFS-347 merge

2013-02-26 Thread Todd Lipcon
=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comme nt-13551755 -- Todd Lipcon Software Engineer, Cloudera

Re: VOTE: HDFS-347 merge

2013-02-20 Thread Todd Lipcon
: -1 The patch seems not ready yet. I have posted some comments/suggestions on the JIRA. Colin also has agreed that there are some bugs to be fixed. Sorry. Tsz-Wo From: Todd Lipcon t...@cloudera.com To: hdfs-dev@hadoop.apache.org Sent: Tuesday

Re: VOTE: HDFS-347 merge

2013-02-20 Thread Todd Lipcon
ready. It seems to have unnecessarily changed the API and protocol. I believe those are important but not trivial things. Tsz-Wo From: Todd Lipcon t...@cloudera.com To: hdfs-dev@hadoop.apache.org; Tsz Wo Sze szets...@yahoo.com Sent: Wednesday, February 20

Re: VOTE: HDFS-347 merge

2013-02-20 Thread Todd Lipcon
and have voted +1. -Todd On Wed, Feb 20, 2013 at 12:16 PM, Todd Lipcon t...@cloudera.com wrote: Hi Nicholas, I looked at your comments on the JIRA, and they all seem like trivial things that could be addressed post-merge, and none of them would affect the functionality. If Colin addresses

Re: VOTE: HDFS-347 merge

2013-02-20 Thread Todd Lipcon
are owned by an HDFS user, and users of the cluster run under other unix credentials. -Todd -- Todd Lipcon Software Engineer, Cloudera

Re: VOTE: HDFS-347 merge

2013-02-20 Thread Todd Lipcon
On Wed, Feb 20, 2013 at 3:06 PM, Todd Lipcon t...@cloudera.com wrote: https://issues.apache.org/jira/browse/HDFS-4476 where I pointed out sorry, meant to link to: https://issues.apache.org/jira/browse/HDFS-2246?focusedCommentId=13102013page=com.atlassian.jira.plugin.system.issuetabpanels:comment

Re: VOTE: HDFS-347 merge

2013-02-20 Thread Todd Lipcon
on a platform which is not yet officially supported on trunk, especially when the old code path is _insecure_ and thus unusable in most environments. Todd From: Todd Lipcon t...@cloudera.com To: hdfs-dev@hadoop.apache.org; Tsz Wo Sze szets...@yahoo.com Sent

Re: VOTE: HDFS-347 merge

2013-02-20 Thread Todd Lipcon
was hoping we could all work together a bit better here... contentious merge votes like this just cause cases where different distros diverge by merging different branches way ahead of the upstream (eg yours with windows support, ours with 347, etc) -Todd -- Todd Lipcon Software Engineer, Cloudera

Re: VOTE: HDFS-347 merge

2013-02-20 Thread Todd Lipcon
that uses raw FileChannel objects. We don't need to cache anything, or make RPCs to the DataNode. from his 10/1/2012 patch upload. So, any patch you might have looked at since then would have clearly removed the old code path. -Todd -- Todd Lipcon Software Engineer, Cloudera

Re: VOTE: HDFS-347 merge

2013-02-19 Thread Todd Lipcon
-tabpanel#comment-13551755 -- Todd Lipcon Software Engineer, Cloudera

[jira] [Resolved] (HDFS-4496) DFSClient: don't create a domain socket unless we need it

2013-02-12 Thread Todd Lipcon (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon resolved HDFS-4496. --- Resolution: Fixed Hadoop Flags: Reviewed DFSClient: don't create a domain socket unless we

[jira] [Created] (HDFS-4485) HDFS-347: DN should chmod socket path a+w

2013-02-08 Thread Todd Lipcon (JIRA)
Todd Lipcon created HDFS-4485: - Summary: HDFS-347: DN should chmod socket path a+w Key: HDFS-4485 URL: https://issues.apache.org/jira/browse/HDFS-4485 Project: Hadoop HDFS Issue Type: Sub-task

[jira] [Created] (HDFS-4486) Add log category for long-running DFSClient notices

2013-02-08 Thread Todd Lipcon (JIRA)
Todd Lipcon created HDFS-4486: - Summary: Add log category for long-running DFSClient notices Key: HDFS-4486 URL: https://issues.apache.org/jira/browse/HDFS-4486 Project: Hadoop HDFS Issue Type

Re: Release numbering for branch-2 releases

2013-02-04 Thread Todd Lipcon
, it wouldn't be wire-protocol-stable. How can we best devise a scheme that explains the various factors above in a more detailed way than one big red warning sticker? What of the above factors does the community think would be implied by GA? Thanks -Todd -- Todd Lipcon Software Engineer, Cloudera

[jira] [Resolved] (HDFS-4433) make TestPeerCache not flaky

2013-01-23 Thread Todd Lipcon (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon resolved HDFS-4433. --- Resolution: Fixed Hadoop Flags: Reviewed make TestPeerCache not flaky

[jira] [Resolved] (HDFS-4416) change dfs.datanode.domain.socket.path to dfs.domain.socket.path

2013-01-21 Thread Todd Lipcon (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon resolved HDFS-4416. --- Resolution: Fixed Hadoop Flags: Reviewed Committed to branch. Thanks, Colin

[jira] [Resolved] (HDFS-4418) HDFS-347: increase default FileInputStreamCache size

2013-01-17 Thread Todd Lipcon (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon resolved HDFS-4418. --- Resolution: Fixed Hadoop Flags: Reviewed Committed to branch. HDFS-347

[jira] [Created] (HDFS-4417) HDFS-347: fix case where local reads get disabled incorrectly

2013-01-16 Thread Todd Lipcon (JIRA)
Todd Lipcon created HDFS-4417: - Summary: HDFS-347: fix case where local reads get disabled incorrectly Key: HDFS-4417 URL: https://issues.apache.org/jira/browse/HDFS-4417 Project: Hadoop HDFS

[jira] [Created] (HDFS-4418) HDFS-347: increase default FileInputStreamCache size

2013-01-16 Thread Todd Lipcon (JIRA)
Todd Lipcon created HDFS-4418: - Summary: HDFS-347: increase default FileInputStreamCache size Key: HDFS-4418 URL: https://issues.apache.org/jira/browse/HDFS-4418 Project: Hadoop HDFS Issue Type

[jira] [Resolved] (HDFS-4400) DFSInputStream#getBlockReader: last retries should ignore the cache

2013-01-14 Thread Todd Lipcon (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon resolved HDFS-4400. --- Resolution: Fixed Hadoop Flags: Reviewed Committed to branch DFSInputStream

[jira] [Resolved] (HDFS-4401) Fix bug in DomainSocket path validation

2013-01-14 Thread Todd Lipcon (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon resolved HDFS-4401. --- Resolution: Fixed Hadoop Flags: Reviewed Committed to branch. Thanks for the fix

[jira] [Resolved] (HDFS-4402) some small DomainSocket fixes: avoid findbugs warning, change log level, etc.

2013-01-14 Thread Todd Lipcon (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon resolved HDFS-4402. --- Resolution: Fixed Hadoop Flags: Reviewed Committed to branch, thanks. some

[jira] [Created] (HDFS-4403) DFSClient can infer checksum type when not provided by reading first byte

2013-01-14 Thread Todd Lipcon (JIRA)
Todd Lipcon created HDFS-4403: - Summary: DFSClient can infer checksum type when not provided by reading first byte Key: HDFS-4403 URL: https://issues.apache.org/jira/browse/HDFS-4403 Project: Hadoop HDFS

[jira] [Resolved] (HDFS-4356) BlockReaderLocal should use passed file descriptors rather than paths

2013-01-11 Thread Todd Lipcon (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon resolved HDFS-4356. --- Resolution: Fixed Hadoop Flags: Reviewed BlockReaderLocal should use passed file

[jira] [Resolved] (HDFS-4388) DomainSocket should throw AsynchronousCloseException when appropriate

2013-01-11 Thread Todd Lipcon (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon resolved HDFS-4388. --- Resolution: Fixed Hadoop Flags: Reviewed Committed to branch. Thanks

[jira] [Resolved] (HDFS-4390) Bypass UNIX domain socket unit tests when they cannot be run

2013-01-11 Thread Todd Lipcon (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon resolved HDFS-4390. --- Resolution: Fixed Hadoop Flags: Reviewed Committed to branch Bypass UNIX

[jira] [Created] (HDFS-4380) Opening a file for read before writer writes a block causes NPE

2013-01-09 Thread Todd Lipcon (JIRA)
Todd Lipcon created HDFS-4380: - Summary: Opening a file for read before writer writes a block causes NPE Key: HDFS-4380 URL: https://issues.apache.org/jira/browse/HDFS-4380 Project: Hadoop HDFS

[jira] [Reopened] (HDFS-4352) Encapsulate arguments to BlockReaderFactory in a class

2013-01-08 Thread Todd Lipcon (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon reopened HDFS-4352: --- By the way, I find it _very_ rude to close someone else's ticket as Invalid or Wont fix without waiting

[jira] [Created] (HDFS-4324) Track and report out-of-date blocks separately from corrupt blocks

2012-12-18 Thread Todd Lipcon (JIRA)
Todd Lipcon created HDFS-4324: - Summary: Track and report out-of-date blocks separately from corrupt blocks Key: HDFS-4324 URL: https://issues.apache.org/jira/browse/HDFS-4324 Project: Hadoop HDFS

[jira] [Created] (HDFS-4305) Add a configurable limit on number of blocks per file, and min block size

2012-12-11 Thread Todd Lipcon (JIRA)
Todd Lipcon created HDFS-4305: - Summary: Add a configurable limit on number of blocks per file, and min block size Key: HDFS-4305 URL: https://issues.apache.org/jira/browse/HDFS-4305 Project: Hadoop HDFS

[jira] [Created] (HDFS-4298) StorageRetentionManager spews warnings when used with QJM

2012-12-10 Thread Todd Lipcon (JIRA)
Todd Lipcon created HDFS-4298: - Summary: StorageRetentionManager spews warnings when used with QJM Key: HDFS-4298 URL: https://issues.apache.org/jira/browse/HDFS-4298 Project: Hadoop HDFS Issue

[jira] [Resolved] (HDFS-3571) Allow EditLogFileInputStream to read from a remote URL

2012-12-05 Thread Todd Lipcon (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon resolved HDFS-3571. --- Resolution: Fixed Fix Version/s: 2.0.3-alpha Committed backport to branch-2. Thanks

[jira] [Resolved] (HDFS-3077) Quorum-based protocol for reading and writing edit logs

2012-12-05 Thread Todd Lipcon (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon resolved HDFS-3077. --- Resolution: Fixed Fix Version/s: 2.0.3-alpha Committed backport to branch-2. Thanks

[jira] [Reopened] (HDFS-4110) Refine JNStorage log

2012-12-05 Thread Todd Lipcon (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon reopened HDFS-4110: --- Reopening to backport to branch-2 Refine JNStorage log

[jira] [Resolved] (HDFS-4110) Refine JNStorage log

2012-12-05 Thread Todd Lipcon (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon resolved HDFS-4110. --- Resolution: Fixed Fix Version/s: 2.0.3-alpha Committed backport to branch-2 (same patch

Re: profiling hdfs write path

2012-12-04 Thread Todd Lipcon
not documented at all, and unit tests are joke. I did some random code changes and some were not detected by unit tests. What methods are you using for testing? If you're just going to insult us, please stay away. We don't need your help unless you're going to be constructive. Todd -- Todd Lipcon

[jira] [Reopened] (HDFS-3571) Allow EditLogFileInputStream to read from a remote URL

2012-12-03 Thread Todd Lipcon (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon reopened HDFS-3571: --- Reopening for merge to branch-2 (this is needed for QJM in branch-2) Allow

Re: HDFS append on upgraded cluster

2012-11-19 Thread Todd Lipcon
, please notify the sender by phone or email immediately and delete it! -- Todd Lipcon Software Engineer, Cloudera

Re: HA for hadoop-0.20.2

2012-11-14 Thread Todd Lipcon
-- Todd Lipcon Software Engineer, Cloudera

[jira] [Created] (HDFS-4176) EditLogTailer should call rollEdits with a timeout

2012-11-12 Thread Todd Lipcon (JIRA)
Todd Lipcon created HDFS-4176: - Summary: EditLogTailer should call rollEdits with a timeout Key: HDFS-4176 URL: https://issues.apache.org/jira/browse/HDFS-4176 Project: Hadoop HDFS Issue Type

[jira] [Created] (HDFS-4169) Add per-disk latency metrics to DataNode

2012-11-08 Thread Todd Lipcon (JIRA)
Todd Lipcon created HDFS-4169: - Summary: Add per-disk latency metrics to DataNode Key: HDFS-4169 URL: https://issues.apache.org/jira/browse/HDFS-4169 Project: Hadoop HDFS Issue Type: Improvement

  1   2   3   4   5   6   7   8   >