Yes thanks Jessica. I re-opened the bug again. I know its not a gui
problem per-say in that the template is not ready to show the download
link. However, it never becomes ready is the actual problem. What sets
the "isready" property to true? As far as I can see, the objects in the
S3 stores (AWS or Cloudian) are complete and from my perspective "ready"
to download/use. It sounds like a bug when registering the template.
Tom.
On Mon, 2013-07-01 at 18:54 +0000, Jessica Wang wrote:
> Thomas,
>
> I checked the data you provided.
>
> The reason that the 2 templates("MyTiny", "AnotherTiny") have no download
> button is because they are not ready
> (i.e. their "isready" property is false).
>
> Download button is only available when "isready" property is true.
>
> Jessica
>
> -----Original Message-----
> From: Thomas O'Dowd [mailto:tpod...@cloudian.com]
> Sent: Thursday, June 27, 2013 8:04 PM
> To: Min Chen
> Cc: dev@cloudstack.apache.org; Jessica Wang
> Subject: Re: Query String Request Authentication(QSRA) support by S3 providers
>
> Hi Min/Jessica,
>
> I attached an image to that issue to show what what my browser is
> showing.
>
> https://issues.apache.org/jira/browse/CLOUDSTACK-3220
>
> Tom.
>
> On Fri, 2013-06-28 at 09:45 +0900, Thomas O'Dowd wrote:
> > Hi Min,
> >
> > Yes. I'll try it again today to check again but when I added Amazon S3
> > as the S3 secondary storage and uploaded a template, I was not shown the
> > "download template" link. However - for Cloudian S3, I am shown it so
> > I'm wondering why.
> >
> > Tom.
> >
> > On Fri, 2013-06-28 at 00:26 +0000, Min Chen wrote:
> > > Hi Tom,
> > >
> > > Are you saying that you cannot see a Download Template button from UI
> > > when Amazon S3 is added as secondary storage? I only tested with RiakCS
> > > and Cloudian, so didn't see this issue. But I am CC Jessica her to confirm
> > > what special handling is done in UI to enable/disable a button from UI.
> > >
> > > Thanks
> > > -min
> > >
> > > On 6/27/13 5:23 PM, "Thomas O'Dowd" <tpod...@cloudian.com> wrote:
> > >
> > > >Hi Min,
> > > >
> > > >Can you check this bug? I'm trying to test this feature for Amazon but
> > > >having no luck getting the Download template link/button to appear.
> > > >
> > > >https://issues.apache.org/jira/browse/CLOUDSTACK-3220
> > > >
> > > >Thanks,
> > > >
> > > >Tom.
> > > >
> > > >On Fri, 2013-06-21 at 17:21 +0000, Min Chen wrote:
> > > >> John,
> > > >>
> > > >> For S3, the api call createEntityExtractUrl is done on
> > > >> management
> > > >>server
> > > >> side; while for NFS secondary storage, if the implementation of
> > > >> createEntityExtractUrl will involve some code be executed in ssvm to
> > > >>copy
> > > >> template from the install location to a public accessible web server
> > > >> location.
> > > >> I don't quite understand some of your comments below. This API
> > > >> is not
> > > >> used to write any information to S3 bucket/directory. This is used for
> > > >> object already existed on S3, and we just provide a URL for user to
> > > >> download a template from S3, just like how Amazon provided user a way
> > > >> to
> > > >> user to extract a S3 object through generatePresignedUrl. We can
> > > >> discuss
> > > >> more on this on collaboration conference.
> > > >>
> > > >> Thanks
> > > >> -min
> > > >>
> > > >>
> > > >>
> > > >> On 6/21/13 7:25 AM, "John Burwell" <jburw...@basho.com> wrote:
> > > >>
> > > >> >Min,
> > > >> >
> > > >> >(I apologize for my belated reply -- I lost track of this draft in the
> > > >> >chaos of the last couple of days.)
> > > >> >
> > > >> >Upon further review, I think I feel into the confusion between
> > > >>management
> > > >> >server and ssvm. This code is executing on the management server
> > > >> >side,
> > > >> >correct? Based on my "corrected" understanding is correct, I would
> > > >>like
> > > >> >to amend my thoughts. Namely, I would like to see the driver
> > > >>operations
> > > >> >pushed out to the SSVM where we can use the stream. As I think about
> > > >>it,
> > > >> >the management server should not need to interact with the driver.
> > > >> >Simply yard up the DataStore attributes + details map and other
> > > >> >extract
> > > >> >parameters, and send them to the SSVM. Using this information, the S3
> > > >> >driver could open a stream to write the template out to the
> > > >> >bucket/directory. I recognize it changes the protocol between the
> > > >> >management server and SSVM, but it simply both sides of the operation
> > > >>by
> > > >> >allowing the DataStore information to be treated opaquely until it is
> > > >> >consumed by the driver to execute the write operation. I also
> > > >>recognize
> > > >> >that we may a little late in the cycle to address it for 4.2, and it
> > > >>may
> > > >> >need to be part of the 4.3 enhancements.
> > > >> >
> > > >> >Thanks,
> > > >> >-John
> > > >> >
> > > >> >On Jun 18, 2013, at 3:55 PM, Min Chen <min.c...@citrix.com> wrote:
> > > >> >
> > > >> >> John,
> > > >> >> In that case, how do we keep backward compatibility of
> > > >>extractTemplate
> > > >> >> api, which requires a URL in the response?
> > > >> >>
> > > >> >> Thanks
> > > >> >> -min
> > > >> >>
> > > >> >> On 6/18/13 11:53 AM, "John Burwell" <jburw...@basho.com> wrote:
> > > >> >>
> > > >> >>> Min,
> > > >> >>>
> > > >> >>> Looking through the code, I think we can simplify driver operation
> > > >>and
> > > >> >>> increase robustness by changing
> > > >> >>>ImageStoreDriver#createEntityExtractUrl()
> > > >> >>> : String to ImageStoreDriver#readEntity(Š) : InputStream. My first
> > > >> >>> concern with the current implementation is that it circumvents any
> > > >> >>> connection pooling/resource management underlying client libraries
> > > >> >>> provide. I/O streams provide a higher-level abstraction that
> > > >> >>> allows
> > > >> >>> drivers to provide the orchestration components with actual
> > > >>resources
> > > >> >>> rather String references. Second, the current interface seems to
> > > >> >>>appears
> > > >> >>> to assume that an http/https URL will be returned. With I/O
> > > >>streams,
> > > >> >>>we
> > > >> >>> can support any client library capable of using the standard I/O
> > > >> >>> framework -- enabling us to support other protocols for downloading
> > > >> >>> templates in the future (e.g. RBD, local filesystem, NBD, etc).
> > > >> >>>
> > > >> >>> Thanks,
> > > >> >>> -John
> > > >> >>>
> > > >> >>> On Jun 18, 2013, at 1:11 PM, Min Chen <min.c...@citrix.com> wrote:
> > > >> >>>
> > > >> >>>> A new version of using generatePresignedUrl in
> > > >>S3ImageStoreDriverImpl
> > > >> >>>>is
> > > >> >>>> checked into object_store.
> > > >> >>>>
> > > >> >>>> THanks
> > > >> >>>> -min
> > > >> >>>>
> > > >> >>>> On 6/18/13 8:29 AM, "Min Chen" <min.c...@citrix.com> wrote:
> > > >> >>>>
> > > >> >>>>> Yes, current code is in
> > > >> >>>>>S3ImageStoreDriverImpl.createEntityExtractUrl,
> > > >> >>>>> which has a security issue mentioned in CLOUDSTACK-3030. I am
> > > >>going
> > > >> >>>>>to
> > > >> >>>>> change it to use generatePresignedUrl api from AWS S3 api.
> > > >> >>>>>
> > > >> >>>>> Thanks
> > > >> >>>>> -min
> > > >> >>>>>
> > > >> >>>>> From: John Burwell
> > > >> >>>>> <jburw...@basho.com<mailto:jburw...@basho.com>>
> > > >> >>>>> Date: Tuesday, June 18, 2013 8:07 AM
> > > >> >>>>> To: Min Chen <min.c...@citrix.com<mailto:min.c...@citrix.com>>
> > > >> >>>>> Cc: Thomas O'Dowd
> > > >> >>>>><tpod...@cloudian.com<mailto:tpod...@cloudian.com>>,
> > > >> >>>>> "dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>"
> > > >> >>>>> <dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>>
> > > >> >>>>> Subject: Re: Query String Request Authentication(QSRA) support by
> > > >>S3
> > > >> >>>>> providers
> > > >> >>>>>
> > > >> >>>>> Min,
> > > >> >>>>>
> > > >> >>>>> Is the code checked into the object_store branch? If so, which
> > > >>lines
> > > >> >>>>> in
> > > >> >>>>> S3TemplateDownloader?
> > > >> >>>>>
> > > >> >>>>> Thanks,
> > > >> >>>>> -John
> > > >> >>>>>
> > > >> >>>>> On Jun 18, 2013, at 12:39 AM, Min Chen
> > > >> >>>>> <min.c...@citrix.com<mailto:min.c...@citrix.com>> wrote:
> > > >> >>>>>
> > > >> >>>>> Hi John,
> > > >> >>>>>
> > > >> >>>>> This is regarding extractTemplate api, where for extractable
> > > >> >>>>>template,
> > > >> >>>>> users can click "Download Template" button from UI to get a http
> > > >>url
> > > >> >>>>>to
> > > >> >>>>> download the template already stored at S3 without providing S3
> > > >> >>>>> credentials. In 4.1, we don't have this issue, since the URL
> > > >>returned
> > > >> >>>>> is
> > > >> >>>>> the public web server location hosted in ssvm, and in 4.2, we are
> > > >> >>>>> returning URL pointing to s3 object. Without setting ACL to the
> > > >> >>>>> S3
> > > >> >>>>> object, user cannot directly click the URL returned from
> > > >> >>>>> extractTemplate
> > > >> >>>>> api to download the template without providing credentials. By
> > > >> >>>>>reading
> > > >> >>>>> the AWS SDK doc today, I ran across the following API that I may
> > > >>be
> > > >> >>>>> able
> > > >> >>>>> to use for this purpose:
> > > >> >>>>>
> > > >> >>>>>
> > > >> >>>>>
> > > >> >>>>>
> > > >>
> > > >>>>>>>URL<http://java.sun.com/j2se/1.5.0/docs/api/java/net/URL.html?is-ext
> > > >>>>>>>er
> > > >> >>>>>na
> > > >> >>>>> l=
> > > >> >>>>> true>
> > > >> >>>>>
> > > >> >>>>>
> > > >>
> > > >>>>>>>generatePresignedUrl<http://docs.aws.amazon.com/AWSJavaSDK/latest/ja
> > > >>>>>>>va
> > > >> >>>>>do
> > > >> >>>>> c/
> > > >> >>>>>
> > > >> >>>>>
> > > >>
> > > >>>>>>>com/amazonaws/services/s3/AmazonS3Client.html#generatePresignedUrl%2
> > > >>>>>>>8j
> > > >> >>>>>av
> > > >> >>>>> a.
> > > >> >>>>>
> > > >> >>>>>
> > > >>
> > > >>>>>>>lang.String,%20java.lang.String,%20java.util.Date,%20com.amazonaws.H
> > > >>>>>>>tt
> > > >> >>>>>pM
> > > >> >>>>> et
> > > >> >>>>>
> > > >> >>>>>
> > > >>
> > > >>>>>>>hod%29>(String<http://java.sun.com/j2se/1.5.0/docs/api/java/lang/Str
> > > >>>>>>>in
> > > >> >>>>>g.
> > > >> >>>>> ht
> > > >> >>>>> ml?is-external=true> bucketName,
> > > >> >>>>>
> > > >> >>>>>
> > > >>
> > > >>>>>>>String<http://java.sun.com/j2se/1.5.0/docs/api/java/lang/String.html
> > > >>>>>>>?i
> > > >> >>>>>s-
> > > >> >>>>> ex
> > > >> >>>>> ternal=true> key,
> > > >> >>>>>
> > > >> >>>>>
> > > >>
> > > >>>>>>>Date<http://java.sun.com/j2se/1.5.0/docs/api/java/util/Date.html?is-
> > > >>>>>>>ex
> > > >> >>>>>te
> > > >> >>>>> rn
> > > >> >>>>> al=true> expiration,
> > > >> >>>>>
> > > >> >>>>>
> > > >>
> > > >>>>>>>HttpMethod<http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/
> > > >>>>>>>am
> > > >> >>>>>az
> > > >> >>>>> on
> > > >> >>>>> aws/HttpMethod.html> method)
> > > >> >>>>> Returns a pre-signed URL for accessing an Amazon S3
> > > >>resource.
> > > >> >>>>>
> > > >> >>>>> This is along the same line as QSRA mentioned by Tom, by wrapped
> > > >>in
> > > >> >>>>> AmazonS3Client for easy consumption. By using this method, I
> > > >> >>>>> think
> > > >> >>>>> that I
> > > >> >>>>> don't need to change ACL of S3 object to open a security hole.
> > > >> >>>>>
> > > >> >>>>> Thanks
> > > >> >>>>> -min
> > > >> >>>>>
> > > >> >>>>> From: John Burwell
> > > >> >>>>> <jburw...@basho.com<mailto:jburw...@basho.com>>
> > > >> >>>>> Date: Monday, June 17, 2013 7:38 PM
> > > >> >>>>> To: Min Chen <min.c...@citrix.com<mailto:min.c...@citrix.com>>
> > > >> >>>>> Cc: Thomas O'Dowd
> > > >> >>>>><tpod...@cloudian.com<mailto:tpod...@cloudian.com>>,
> > > >> >>>>> "dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>"
> > > >> >>>>> <dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>>
> > > >> >>>>> Subject: Re: Query String Request Authentication(QSRA) support by
> > > >>S3
> > > >> >>>>> providers
> > > >> >>>>>
> > > >> >>>>> Min,
> > > >> >>>>>
> > > >> >>>>> Why are we mucking with ACLs at all? The best security practice
> > > >> >>>>>would
> > > >> >>>>> be
> > > >> >>>>> to create a bucket for CloudStack's use and assign it a dedicated
> > > >> >>>>> access
> > > >> >>>>> key and secret key pair with read/write access only to that
> > > >>bucket.
> > > >> >>>>> Requiring an administrative account to an object store opens an
> > > >> >>>>> unnecessarily large attack surface. Therefore, as implemented in
> > > >> >>>>>4.1,
> > > >> >>>>> we
> > > >> >>>>> should defer bucket creation, ACL assignment, and credential
> > > >>creation
> > > >> >>>>> to
> > > >> >>>>> the administrator/operator.
> > > >> >>>>>
> > > >> >>>>> Thanks,
> > > >> >>>>> -John
> > > >> >>>>>
> > > >> >>>>> On Jun 17, 2013, at 1:15 PM, Min Chen
> > > >> >>>>> <min.c...@citrix.com<mailto:min.c...@citrix.com>> wrote:
> > > >> >>>>>
> > > >> >>>>> Tom filed a very good bug for ACL setting change on S3 object
> > > >> >>>>> when
> > > >> >>>>> users
> > > >> >>>>> issue extractTemplate API
> > > >> >>>>> (https://issues.apache.org/jira/browse/CLOUDSTACK-3030), and his
> > > >> >>>>> recommendation of using Query String Request Authentication
> > > >> >>>>> (QSRA)
> > > >> >>>>> alternative sounds like a right approach to fix this bug. Before
> > > >> >>>>> implementing it, I would like to confirm if QSRA should be
> > > >>supported
> > > >> >>>>>by
> > > >> >>>>> all S3 providers if they claim that they are AWS s3 compatible.
> > > >> >>>>> If
> > > >> >>>>>so,
> > > >> >>>>> we
> > > >> >>>>> will make this assumption in our code. Based on Tom, Cloudian is
> > > >> >>>>> supporting it. How about RiakCS, John?
> > > >> >>>>>
> > > >> >>>>> Thanks
> > > >> >>>>> -min
> > > >> >>>>>
> > > >> >>>>>
> > > >> >>>>
> > > >> >>>
> > > >> >>
> > > >> >
> > > >>
> > > >
> > > >--
> > > >Cloudian KK - http://www.cloudian.com/get-started.html
> > > >Fancy 100TB of full featured S3 Storage?
> > > >Checkout the Cloudian(R) Community Edition!
> > > >
> > >
> >
>
--
Cloudian KK - http://www.cloudian.com/get-started.html
Fancy 100TB of full featured S3 Storage?
Checkout the Cloudian® Community Edition!
