[jupyter] Easiest way to run code from your jupyterlab notebook cell in the Cloud

Hello All, Here's a super easy way to take code from your Jupyterlab notebook cell and run it in the Cloud. This is a feature of our free service InfinStor Starter. Here's how it works: - Write your python code in a jupyterlab notebook cell - Capture the code and its environment (conda o

Re: Hadoop - Major releases

Thichphuoctien On Mar 9, 2015 3:35 PM, "Andrew Wang" wrote: > Hi Mayank, > > Note that Hadoop 3 does not mean the end of updates for Hadoop 2.x, which > will keep supporting JDK7 for a while yet. Someone on the original thread > also proposed keeping Hadoop 3 JDK7-source compatible to make backpo

RE: 2.1.2 (Was: Re: [VOTE] Release Apache Hadoop 2.1.1-beta)

+1 Makes good sense. Jagane -Original Message- From: Arun C Murthy [mailto:a...@hortonworks.com] Sent: Tuesday, October 01, 2013 4:15 PM To: common-dev@hadoop.apache.org; hdfs-...@hadoop.apache.org; yarn-...@hadoop.apache.org; mapreduce-...@hadoop.apache.org Subject: 2.1.2 (Was: Re: [VO

[jira] [Commented] (HDFS-4858) HDFS DataNode to NameNode RPC should timeout

[ https://issues.apache.org/jira/browse/HDFS-4858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13737264#comment-13737264 ] Jagane Sundar commented on HDFS-4858: - Manually, this is quite easy to simulat

[jira] [Commented] (HDFS-4858) HDFS DataNode to NameNode RPC should timeout

[ https://issues.apache.org/jira/browse/HDFS-4858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13733002#comment-13733002 ] Jagane Sundar commented on HDFS-4858: - I am concerned that we might not even be

[jira] [Updated] (HDFS-4858) HDFS DataNode to NameNode RPC should timeout

[ https://issues.apache.org/jira/browse/HDFS-4858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jagane Sundar updated HDFS-4858: Attachment: HDFS-4858.patch > HDFS DataNode to NameNode RPC should time

[jira] [Updated] (HDFS-4858) HDFS DataNode to NameNode RPC should timeout

[ https://issues.apache.org/jira/browse/HDFS-4858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jagane Sundar updated HDFS-4858: Status: Patch Available (was: Open) > HDFS DataNode to NameNode RPC should time

[jira] [Commented] (HADOOP-9654) IPC timeout doesn't seem to be kicking in

[ https://issues.apache.org/jira/browse/HADOOP-9654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13688652#comment-13688652 ] Jagane Sundar commented on HADOOP-9654: --- Roman - pardon me if you already

[jira] [Commented] (HDFS-4849) Idempotent create and append operations.

[ https://issues.apache.org/jira/browse/HDFS-4849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13686126#comment-13686126 ] Jagane Sundar commented on HDFS-4849: - Suresh - I hope I am answering the cor

[jira] [Commented] (HDFS-4849) Idempotent create and append operations.

[ https://issues.apache.org/jira/browse/HDFS-4849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13686089#comment-13686089 ] Jagane Sundar commented on HDFS-4849: - If I may chime in here: I agree that the

[jira] [Created] (HDFS-4858) HDFS DataNode to NameNode RPC should timeout

Jagane Sundar created HDFS-4858: --- Summary: HDFS DataNode to NameNode RPC should timeout Key: HDFS-4858 URL: https://issues.apache.org/jira/browse/HDFS-4858 Project: Hadoop HDFS Issue Type: Bug

[jira] [Created] (HDFS-4858) HDFS DataNode to NameNode RPC should timeout

Jagane Sundar created HDFS-4858: --- Summary: HDFS DataNode to NameNode RPC should timeout Key: HDFS-4858 URL: https://issues.apache.org/jira/browse/HDFS-4858 Project: Hadoop HDFS Issue Type: Bug

RE: [PROPOSAL] change in bylaws to remove Release Plan vote

! :-) Thanks, --Matt On Wed, May 22, 2013 at 3:52 AM, Steve Loughran wrote: > On 21 May 2013 23:47, Jagane Sundar wrote: > > > I see one significant benefit to having Release Plan votes: Fewer > releases > > with more members of the community working on any given releas

RE: [PROPOSAL] change in bylaws to remove Release Plan vote

I see one significant benefit to having Release Plan votes: Fewer releases with more members of the community working on any given release. In turn, fewer Hadoop releases implies less confusion for end users attempting to download and use an Apache Hadoop release. If there are a dozen different

RE: [VOTE] Release plan for Hadoop 2.0.5

+1 (non-binding) for adopting the 'fix bugs and include all changes required for freezing APIs, but defer invasive feature changes to a follow on release' (I would like to preface my comments by stating that I have a great deal of respect and admiration for the folks who created and developed Ha

[jira] [Commented] (HDFS-4269) DatanodeManager#registerDatanode rejects all datanode registrations from localhost in single-node developer setup

[ https://issues.apache.org/jira/browse/HDFS-4269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13638563#comment-13638563 ] Jagane Sundar commented on HDFS-4269: - OK. The TCP endpoint gives you an IP add

[jira] [Commented] (HDFS-4269) DatanodeManager#registerDatanode rejects all datanode registrations from localhost in single-node developer setup

[ https://issues.apache.org/jira/browse/HDFS-4269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13631519#comment-13631519 ] Jagane Sundar commented on HDFS-4269: - I apologize - I should have read the

[jira] [Commented] (HDFS-4269) DatanodeManager#registerDatanode rejects all datanode registrations from localhost in single-node developer setup

[ https://issues.apache.org/jira/browse/HDFS-4269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13631498#comment-13631498 ] Jagane Sundar commented on HDFS-4269: - Thanks for your explanation, Chris.

[jira] [Commented] (HDFS-3990) NN's health report has severe performance problems

[ https://issues.apache.org/jira/browse/HDFS-3990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13631357#comment-13631357 ] Jagane Sundar commented on HDFS-3990: - Here is another instance of this problem.

[jira] [Commented] (HDFS-4269) DatanodeManager#registerDatanode rejects all datanode registrations from localhost in single-node developer setup

[ https://issues.apache.org/jira/browse/HDFS-4269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13631355#comment-13631355 ] Jagane Sundar commented on HDFS-4269: - It would appear that there is more to

[jira] [Commented] (HDFS-4646) createNNProxyWithClientProtocol ignores configured timeout value

[ https://issues.apache.org/jira/browse/HDFS-4646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13618618#comment-13618618 ] Jagane Sundar commented on HDFS-4646: - I uploaded another patch with indenta

[jira] [Updated] (HDFS-4646) createNNProxyWithClientProtocol ignores configured timeout value

[ https://issues.apache.org/jira/browse/HDFS-4646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jagane Sundar updated HDFS-4646: Attachment: HDFS-4646.001.patch > createNNProxyWithClientProtocol ignores configured time

[jira] [Commented] (HDFS-4646) createNNProxyWithClientProtocol ignores configured timeout value

[ https://issues.apache.org/jira/browse/HDFS-4646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13618119#comment-13618119 ] Jagane Sundar commented on HDFS-4646: - Regarding writing a unit test for this patc

[jira] [Updated] (HDFS-4646) createNNProxyWithClientProtocol ignores configured timeout value

[ https://issues.apache.org/jira/browse/HDFS-4646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jagane Sundar updated HDFS-4646: Attachment: HDFS-4646.patch > createNNProxyWithClientProtocol ignores configured timeout va

[jira] [Updated] (HDFS-4646) createNNProxyWithClientProtocol ignores configured timeout value

[ https://issues.apache.org/jira/browse/HDFS-4646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jagane Sundar updated HDFS-4646: Status: Patch Available (was: Open) > createNNProxyWithClientProtocol ignores configu

[jira] [Updated] (HDFS-4646) createNNProxyWithClientProtocol ignores configured timeout value

[ https://issues.apache.org/jira/browse/HDFS-4646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jagane Sundar updated HDFS-4646: Status: Open (was: Patch Available) > createNNProxyWithClientProtocol ignores configu

[jira] [Updated] (HDFS-4646) createNNProxyWithClientProtocol ignores configured timeout value

[ https://issues.apache.org/jira/browse/HDFS-4646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jagane Sundar updated HDFS-4646: Status: Patch Available (was: Open) > createNNProxyWithClientProtocol ignores configu

[jira] [Created] (HDFS-4646) createNNProxyWithClientProtocol ignores configured timeout value

Jagane Sundar created HDFS-4646: --- Summary: createNNProxyWithClientProtocol ignores configured timeout value Key: HDFS-4646 URL: https://issues.apache.org/jira/browse/HDFS-4646 Project: Hadoop HDFS

[jira] [Created] (HDFS-4646) createNNProxyWithClientProtocol ignores configured timeout value

Jagane Sundar created HDFS-4646: --- Summary: createNNProxyWithClientProtocol ignores configured timeout value Key: HDFS-4646 URL: https://issues.apache.org/jira/browse/HDFS-4646 Project: Hadoop HDFS

[jira] [Commented] (HDFS-2802) Support for RW/RO snapshots in HDFS

[ https://issues.apache.org/jira/browse/HDFS-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13491083#comment-13491083 ] Jagane Sundar commented on HDFS-2802: - Adding the capability for a user to set his

[jira] [Commented] (HDFS-2802) Support for RW/RO snapshots in HDFS

[ https://issues.apache.org/jira/browse/HDFS-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13490083#comment-13490083 ] Jagane Sundar commented on HDFS-2802: - First off, I think this is a critical fea

[jira] [Commented] (HDFS-3370) HDFS hardlink

[ https://issues.apache.org/jira/browse/HDFS-3370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13454144#comment-13454144 ] Jagane Sundar commented on HDFS-3370: - Thanks for the pointer to HBASE-6055, Jess

[jira] [Commented] (HDFS-3370) HDFS hardlink

[ https://issues.apache.org/jira/browse/HDFS-3370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13453183#comment-13453183 ] Jagane Sundar commented on HDFS-3370: - Pardon my naive question - but, are hard l

RE: [DISCUSS] Spin out MR, HDFS and YARN as their own TLPs and disband Hadoop umbrella project

> As for technical things we need to do to get to a feasible split: big > +1 that classpath pollution issues are near top of the list. We need a > reasonable classloader strategy, and I think Tom's OSGi stuff is a > good start in that direction. But it's going to be quite some time > before that's

[libvirt] Support for pass through lines in domain XML?

Hello All, I have a question regarding the domain XML and its conversion to hypervisor specific config files. I am using libvirt to talk to vmware ESX. ESX has a feature where you can add a line such as: guestinfo.xxx=abc in the .vmx file that describes the VM. Later, the guest can extra

RE: why hadoop choose maven as its build tool?

Hallelujah. I thought I was the only one who felt that maven's non-deterministic result was just plain batty. In this mavenized world, is it guaranteed that if I build twice using maven on two different machines in two different locations, I will get the same exact bits? If this guarantee exi

RE: [VOTE] Rename hadoop branches post hadoop-1.x

1,2,4,3,5 Jagane On Mon, Mar 19, 2012 at 06:03PM, Arun C Murthy wrote: > My sincere apologies. I messed up on option (4) - it should read: > > > (4) If security is fixed in branch-0.22 within a short time-frame i.e. 2 > > months then we get option 1, else we get option 3. Effectively postpone >

[jira] [Commented] (HADOOP-8136) Enhance hadoop to use a newer version (0.8.1) of the jets3t library

[ https://issues.apache.org/jira/browse/HADOOP-8136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13232297#comment-13232297 ] Jagane Sundar commented on HADOOP-8136: --- I am working on the patch for trunk,

RE: Need help with getting some patches for 1.0.2 reviewed

From: Suresh Srinivas [sur...@hortonworks.com] Sent: Friday, March 16, 2012 12:12 PM To: common-dev@hadoop.apache.org Subject: Re: Need help with getting some patches for 1.0.2 reviewed On Tue, Mar 6, 2012 at 10:40 AM, Jagane Sundar wrote: > Hey folks, > >

RE: [PLAN] Freeze date, Hadoop 1.0.2 RC

Hey Matt, I would like to propose that we include a fix for the following bug: https://issues.apache.org/jira/browse/HADOOP-8136 This requires a backport of the following fix: https://issues.apache.org/jira/browse/HADOOP-5836 Thanks, Jagane -Original Message- From: Matt Foley [mailto:

Need help with getting some patches for 1.0.2 reviewed

Hey folks, I have a couple of patches to the 1.0 branch that I would like to shepherd through the process. I'm not entirely familiar with the process, so bear with me.. - I have a backport of HADOOP-5836 for the 1.0 branch that I have uploaded to jira. This is a no-code-change, just a re-spin

[jira] [Updated] (HADOOP-8136) Enhance hadoop to use a newer version (0.8.1) of the jets3t library

[ https://issues.apache.org/jira/browse/HADOOP-8136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jagane Sundar updated HADOOP-8136: -- Attachment: HADOOP-8136-0-for_branch_1_0.patch Apply the patch HADOOP-5836-for_branch_1_0

[jira] [Updated] (HADOOP-8136) Enhance hadoop to use a newer version (0.8.1) of the jets3t library

[ https://issues.apache.org/jira/browse/HADOOP-8136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jagane Sundar updated HADOOP-8136: -- Target Version/s: 1.0.2 Status: Patch Available (was: Open) Here's my

RE: Some s3n improvements to the 1.0 branch? Also, is there a 1.1.0 planned?

e 1.0 branch? Also, is there a 1.1.0 planned? Apologies for being pedantic: please set the target-version to 1.0.2. The committer/RM set the fix-version after it's merged into the release branch. thanks, Arun On Mar 5, 2012, at 12:45 PM, Jagane Sundar wrote: > Thanks, Eli. > > I have

RE: Some s3n improvements to the 1.0 branch? Also, is there a 1.1.0 planned?

em in for the next dot release. Thanks, Eli On Mon, Mar 5, 2012 at 9:47 AM, Jagane Sundar wrote: > Hello All, > > I would like to make some enhancements to the s3n driver in branch-1.0. > > Specifically, I would like to accomplish the following: > 1. Backport HADOOP-5836 > 2.

[jira] [Commented] (HADOOP-5836) Bug in S3N handling of directory markers using an object with a trailing "/" causes jobs to fail

[ https://issues.apache.org/jira/browse/HADOOP-5836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13222597#comment-13222597 ] Jagane Sundar commented on HADOOP-5836: --- A note on running the unit test for

[jira] [Updated] (HADOOP-5836) Bug in S3N handling of directory markers using an object with a trailing "/" causes jobs to fail

[ https://issues.apache.org/jira/browse/HADOOP-5836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jagane Sundar updated HADOOP-5836: -- Attachment: HADOOP-5836-0-for_branch_1_0.patch Re-spin of this patch for branch-1.0. Code is

Some s3n improvements to the 1.0 branch? Also, is there a 1.1.0 planned?

Hello All, I would like to make some enhancements to the s3n driver in branch-1.0. Specifically, I would like to accomplish the following: 1. Backport HADOOP-5836 2. Update the jets3t library from 0.6.1 to 0.8.1. This entails code changes in the s3n driver to track API changes in jets3t version

[jira] [Created] (HADOOP-8136) Enhance hadoop to use a newer version (0.8.1) of the jets3t library

Type: Improvement Components: fs/s3 Affects Versions: 1.0.0, 0.22.0, 0.23.3 Environment: Ubuntu 11.04, 64 bit, JDK 1.6.0_30 Reporter: Jagane Sundar Hadoop is built against, and includes, an older version of the Jets3t library - version 0.6.1. The current version of

[jira] [Created] (HADOOP-8136) Enhance hadoop to use a newer version (0.8.1) of the jets3t library

Type: Improvement Components: fs/s3 Affects Versions: 1.0.0, 0.22.0, 0.23.3 Environment: Ubuntu 11.04, 64 bit, JDK 1.6.0_30 Reporter: Jagane Sundar Hadoop is built against, and includes, an older version of the Jets3t library - version 0.6.1. The current version of

Detecting HBase cluster idle

Hello HBase gurus, I have a question - can I use the metric 'Cluster Requests (R)' published by the HBase Master's metrics system to detect whether a cluster is idle or not? Upon first blush this seems like a good idea, but without knowledge of HBase internals, I am not sure that there aren't a

RE: Writing the WAL to a different filesystem from the HFiles

> Not always being able to read back an object that has been written is deadly. > Having the S3 client cache written > data for a while can help but isn't a > complete solution because the RS can fail and its regions will be reassigned > > to another RS... who then might not be able to read the

RE: Writing the WAL to a different filesystem from the HFiles

Hello Andy, >> No, definitely not full object reads, we use HDFS positioned reads, which >> allow us to request, within a gigabyte plus store file, much smaller byte >> >> ranges (e.g. 64 KB), and receive back only the requested data. We can "seek" >> around the file. Ahh. This is good to kno

RE: Writing the WAL to a different filesystem from the HFiles

Stack" wrote: > On Mon, Dec 26, 2011 at 8:06 PM, Jagane Sundar wrote: > > Is this a reasonable thing to attempt? Or am I out of my mind? > > > > Check back in this mailing list archive for opinions on hbase atop s3. > St.Ack >

Writing the WAL to a different filesystem from the HFiles

Hello HBaseistas, I have a question: Has anyone here tried to run HBase with the WAL going to one FileSystem, and the HFiles going to another? My casual perusal of the source code indicates that this is not possible, so I am willing to hack a configuration option to do this and try it out. Let

Re: Hadoop as a Big Data app for the cloud

On Thu, Oct 6, 2011 at 11:21 AM, Daniel Sikar wrote: > > If you buy the argument that EBS is resilient storage > > Just for the record, data has been lost in EBS. > > Right. That's why I qualified the statement with 'if you buy the argument...'. >From Amazon's website: 'The durability of your vo

Hadoop as a Big Data app for the cloud

Note that I have changed the subject to be more relevant. > > I've just started the wiki page on this topic: > http://wiki.apache.org/hadoop/**Virtual%20Hadoop > > I will take a look at this wiki, and hopefully, contribute to it, Steve. > > > The

Re: Which proposed distro of Hadoop, 0.20.206 or 0.22, will be better for HBase?

On Wed, Oct 5, 2011 at 10:09 PM, Konstantin Boudnik wrote: > On Wed, Oct 05, 2011 at 07:00PM, Jagane Sundar wrote: > > approaches you are familiar with. Chef/Puppet et. al. are not interesting > to > > Is this a technical lack of interest as in these solutions do not perform

Re: Which proposed distro of Hadoop, 0.20.206 or 0.22, will be better for HBase?

Thanks for your input, Milind. It's very useful and interesting. In the interest of brevity, I have truncated most of it except for the point regarding 'cloud friendly'. I have done some research into this, and want to get some more community feedback. >2. Built in support for the cloud. (Whirr i

Re: Which proposed distro of Hadoop, 0.20.206 or 0.22, will be better for HBase?

on, past or > present, the author might be affiliated with.) > > > > On 10/2/11 4:57 PM, "Jagane Sundar" wrote: > > >Hello Hadoop experts, > > > >I would like to solicit your input in answering this question. Which > >proposed distro of Hadoop, 0.20.206 or 0

Which proposed distro of Hadoop, 0.20.206 or 0.22, will be better for HBase?

Hello Hadoop experts, I would like to solicit your input in answering this question. Which proposed distro of Hadoop, 0.20.206 or 0.22, is likely to be the better platform for hosting HBase? My requirements are as follows: 1. The Hadoop must support both HBase and MR jobs in the same cluster

Which proposed distro of Hadoop, 0.20.206 or 0.22, will be better for HBase?

Hello Hadoop experts, I would like to solicit your input in answering this question. Which proposed distro of Hadoop, 0.20.206 or 0.22, is likely to be the better platform for hosting HBase? My requirements are as follows: 1. The Hadoop must support both HBase and MR jobs in the same cluster. At

Re: [libvirt] RFC: API additions for enhanced snapshot support

On 7/5/2011 2:36 PM, Eric Blake wrote: On 07/04/2011 07:53 PM, Jagane Sundar wrote: Thanks for looping me in, Stefan. Does each volume have its own independent snapshot namespace? It may be wise to document that snapshot namespaces are *not* independent because storage backends may not be

Re: [libvirt] RFC: API additions for enhanced snapshot support

Thanks for looping me in, Stefan. Does each volume have its own independent snapshot namespace? It may be wise to document that snapshot namespaces are *not* independent because storage backends may not be able to provide these semantics. There is a need for 'just-a-volume-snapshot', and for

[jira] [Commented] (HADOOP-7417) Hadoop Management System (Umbrella)

[ https://issues.apache.org/jira/browse/HADOOP-7417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13056315#comment-13056315 ] Jagane Sundar commented on HADOOP-7417: --- The goal of HMS is to provide a Ha

[jira] [Commented] (HADOOP-7417) Hadoop Management System (Umbrella)

[ https://issues.apache.org/jira/browse/HADOOP-7417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13056217#comment-13056217 ] Jagane Sundar commented on HADOOP-7417: --- Let me expand a little bit on Er

Re: [Qemu-devel] [patch 6/7] QEMU live block copy

Hello Stefan, Can you expand on this some more? I have similar concerns for Livebackup. At the beginning of your paragraph, did you mean 'asynchronous I/O emulation' instead of 'synchronous I/O emulation'? Also, I don't understand the 'stack' construct that you refer to. When you say 'push a

Re: [Qemu-devel] [patch 6/7] QEMU live block copy

On 6/7/2011 5:15 AM, Stefan Hajnoczi wrote: On Mon, Jun 6, 2011 at 5:55 PM, Marcelo Tosatti wrote: I haven't reviewed this whole patch yet, but comments below. This patch, like image streaming, may hit deadlocks due to synchronous I/O emulation. I discovered this problem when working on image

Re: [libvirt] [Qemu-devel] [RFC] live snapshot, live merge, live block migration

On 5/27/2011 9:46 AM, Stefan Hajnoczi wrote: On Mon, May 23, 2011 at 2:02 PM, Stefan Hajnoczi wrote: On Sun, May 22, 2011 at 10:52 AM, Dor Laor wrote: On 05/20/2011 03:19 PM, Stefan Hajnoczi wrote: I'm interested in what the API for snapshots would look like. Specifically how does user softw

Re: [Qemu-devel] [RFC] live snapshot, live merge, live block migration

On 5/27/2011 9:46 AM, Stefan Hajnoczi wrote: On Mon, May 23, 2011 at 2:02 PM, Stefan Hajnoczi wrote: On Sun, May 22, 2011 at 10:52 AM, Dor Laor wrote: On 05/20/2011 03:19 PM, Stefan Hajnoczi wrote: I'm interested in what the API for snapshots would look like. Specifically how does user softw

Re: [Qemu-devel] [RFC] live snapshot, live merge, live block migration

Hello Stefan, I have been thinking about this since you sent out this message. A quick look at the libvirt API indicates that their notion of a snapshot often refers to a "disk+memory snapshot". It would be good to provide feedback to the libvirt developers to make sure that proper support for a

Re: [Qemu-devel] [RFC] live snapshot, live merge, live block migration

ts in these three technologies, I think we should support all three in base qemu. This would make qemu/qemu-kvm more feature rich than vmware, xen and hyper-v. Thanks, Jagane On 5/17/2011 3:53 PM, Dor Laor wrote: On 05/16/2011 11:23 AM, Jagane Sundar wrote: Hello Dor, Let me see if I understand

Re: [Qemu-devel] [RFC] live snapshot, live merge, live block migration

ts in these three technologies, I think we should support all three in base qemu. This would make qemu/qemu-kvm more feature rich than vmware, xen and hyper-v. Thanks, Jagane On 5/17/2011 3:53 PM, Dor Laor wrote: On 05/16/2011 11:23 AM, Jagane Sundar wrote: Hello Dor, Let me see if I understand

Re: [Qemu-devel] [RFC] live snapshot, live merge, live block migration

Hello Dor, Let me see if I understand live snapshot correctly: If I want to configure a VM for daily backup, then I would do the following: - Create a snapshot s1. s0 is marked read-only. - Do a full backup of s0 on day 0. - On day 1, I would create a new snapshot s2, then copy over the snapsho

Re: [Qemu-devel] [RFC] live snapshot, live merge, live block migration

Hello Dor, Let me see if I understand live snapshot correctly: If I want to configure a VM for daily backup, then I would do the following: - Create a snapshot s1. s0 is marked read-only. - Do a full backup of s0 on day 0. - On day 1, I would create a new snapshot s2, then copy over the snapsho

Re: [Qemu-devel] [RFC] live snapshot, live merge, live block migration

Hello Dor, One important advantage of live snapshot over live backup is support of multiple (consecutive) live snapshots while there can be only a single live backup at one time. This is why I tend to think that although live backup carry some benefit (no merge required), the live snapshot + liv

Re: [Qemu-devel] [RFC] live snapshot, live merge, live block migration

Hello Dor, One important advantage of live snapshot over live backup is support of multiple (consecutive) live snapshots while there can be only a single live backup at one time. This is why I tend to think that although live backup carry some benefit (no merge required), the live snapshot + liv

Re: snapmirror functionality using qcow2 and rsync?

Hello Fred, As Stefan points out, there is a lot of interest in such functionality. I am the author of a feature called Livebackup - which is aimed at solving exactly this problem - backup of live virtual machines. The additional constraint I place is that backup of the VM should be enabled rega

Re: [Qemu-devel] [RFC] live snapshot, live merge, live block migration

On 5/12/2011 8:33 AM, Jes Sorensen wrote: On 05/09/11 15:40, Dor Laor wrote: Summary: * We need Marcelo's new (to come) block copy implementation * should work in parallel to migration and hotplug * General copy on read is desirable * Live snapshot merge to be implemented using blo

More work on Livebackup for qemu/qemu-kvm

Hello All, I have made more progress on the proposed Livebackup feature for qemu and qemu-kvm. Based on Jes' feedback, I have switched over to using command line parameters instead of specific named files. So, a typical command line looks like this: # ./x86_64-softmmu/qemu-system-x86_64 -drive

Re: [Openstack] lunr reference iSCSI target driver

Hello Mike, Tomo: I invite you to take a look at Livebackup for kvm: http://wiki.qemu.org/Features/Livebackup This is a feature that I am currently developing. In brief, livebackup enables full and incremental backups of running VMs. It has enhancements to qemu that enable it to keep an in-memo

Re: Livebackup feature for qemu/qemu-kvm

On 5/2/2011 11:59 PM, Jes Sorensen wrote: On 05/03/11 07:58, Jagane Sundar wrote: On 5/2/2011 7:36 AM, Jes Sorensen wrote: Reading your page, the first thing I stumble upon under 'Use Cases' is the reference to EBS storage. What is that? EBS stands for Elastic Block Storage - Am

Re: Livebackup feature for qemu/qemu-kvm

On 5/2/2011 7:36 AM, Jes Sorensen wrote: On 05/02/11 00:23, Jagane Sundar wrote: Having a live backup feature for kvm will make it a better solution for IaaS clouds compared to xen. I would like to solicit feedback from all of you folks involved in the block subsystem of qemu. Stefan mentioned

Livebackup feature for qemu/qemu-kvm

Hello All, I have been working on a feature called Livebackup for qemu-kvm (actually it is applicable to qemu). This is a feature that enables a system administrator or a backup program to take full and incremental backups of running VMs. It is explained in more detail here: http://wiki.qemu

Livebackup wiki page on wiki.qemu.org now available

It is available at: http://wiki.qemu.org/Features/Livebackup It is a work in progress, and I will continue to add control flow diagrams, and other illustrative content over the next week or so. Feedback is welcome. Thanks, Jagane -- To unsubscribe from this list: send the line "unsubscribe kvm

Re: A Live Backup feature for KVM

On 4/25/2011 6:34 AM, Stefan Hajnoczi wrote: On Mon, Apr 25, 2011 at 9:16 AM, Jagane Sundar wrote: The direction that I chose to go is slightly different. In both of the proposals you pointed me at, the original virtual disk is made read-only and the VM writes to a different COW file. After

Re: A Live Backup feature for KVM

ing over the next few weeks. I will also create a detailed proposal on wiki.qemu.org, with much more information. Thanks, Jagane On 4/24/2011 1:32 AM, Stefan Hajnoczi wrote: On Sun, Apr 24, 2011 at 12:17 AM, Jagane Sundar wrote: I would like to get your input on a KVM feature that I am currentl

A Live Backup feature for KVM

ted backups scheduled for instances created using an EC2 or Openstack API. Please let me know if you find this feature interesting. I am looking forward to feedback on any and all aspects of this design. I would like to work with the KVM community to contribute this feature to the KVM code

[jira] [Commented] (MAPREDUCE-2413) TaskTracker should handle disk failures at both startup and runtime

[ https://issues.apache.org/jira/browse/MAPREDUCE-2413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13021871#comment-13021871 ] Jagane Sundar commented on MAPREDUCE-2413: -- Owen - I have made the com

[jira] [Updated] (MAPREDUCE-2413) TaskTracker should handle disk failures at both startup and runtime

[ https://issues.apache.org/jira/browse/MAPREDUCE-2413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jagane Sundar updated MAPREDUCE-2413: - Attachment: MR-2413.v0.2.patch > TaskTracker should handle disk failures at b

[jira] [Commented] (MAPREDUCE-2413) TaskTracker should handle disk failures at both startup and runtime

[ https://issues.apache.org/jira/browse/MAPREDUCE-2413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13021851#comment-13021851 ] Jagane Sundar commented on MAPREDUCE-2413: -- >> Why d

Re: [Openstack] distributed and heterogeneous schedulers

_ From: openstack-bounces+jagane=sundar@lists.launchpad.net [openstack-bounces+jagane=sundar@lists.launchpad.net] On Behalf Of Sandy Walsh [sandy.wa...@rackspace.com] Sent: Friday, April 15, 2011 2:24 PM To: Jay Pipes; Ed Leafe Cc: Mark Washenberger; openstack@lists.launchpad.net S

[Openstack] Any progress on RequestCapabilities? Can I contribute?

Hello all, I was wondering if there has been any progress on the following proposal: http://wiki.openstack.org/RequestCapabilities I am interested in implementing 'thin provisioning' capability using the framework outlined at the above webpage. This will be for KVM Virtualization initially, pe