Re: [Openstack] [Swift] Where's Object's metadata located ?
On 18/06/12 11:57, Kuo Hugo wrote: > Hi Adrian , > > Thanks for your explanation ... > > About Q2 , manifest question > Is there any audit mechanism to delete segments of failure uploading object? > What if the uploading procedure is been interrupted by user . > As you said , I think the segments still available for accessing . > On the other hand , it means that those segmented objects will keep on > disk in object-server , even though the uploading failure is caused by > user him self. > Those segments are regular files. The big file support it's just uploading the big file split in parts smaller than 5GB + a manifest file that points to the place of the parts. The user can both access to the parts (that are regular files) and to the manifest, that basically re-assembles the parts automatically on the fly. Kind regards, Juan ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Swift] Where's Object's metadata located ?
Hi Adrian , Thanks for your explanation ... About Q2 , manifest question Is there any audit mechanism to delete segments of failure uploading object? What if the uploading procedure is been interrupted by user . As you said , I think the segments still available for accessing . On the other hand , it means that those segmented objects will keep on disk in object-server , even though the uploading failure is caused by user him self. Is there any approach to remove those segmented objects? or it might waste some disk space. Thanks 2012/6/17 Adrian Smith > > Q1: Where's the metadata of an object ? > It's stored in extended attributes on the filesystem itself. This is > reason XFS (or other filesystem supporting extended attributes) is > required. > > > Could I find the value? > Sure. You just need some way of a) identifying the object on disk and, > b) a means of querying the extended metadata (using for example the > python xattrs package). > > > Q2: What if a large object be interrupted during upload , will the > manifest objects be deleted? > Large objects (i.e. those > 5Gb) must be split up client-side and the > segments uploaded individually. When all the segments are uploaded the > manifest must then be created by the client. What I'm trying to get at > is that each segment and even the manifest are completely independent > objects. A failure during the upload of any one segment has no impact > on other segments or the manifest. > > Adrian > > > On 16 June 2012 09:53, Kuo Hugo wrote: > > Hi folks , > > > > Q1: > > Where's the metadata of an object ? > > For example the "X-Object-Manifest". Does it store in inode ? > > I did not see the matadata "X-Object=Manifest" in container's DB. > > > > Could I find the value? > > > > Q2: > > What if a large object be interrupted during upload , will the manifest > > objects be deleted? > > For example , > > OBJ1 :200MB > > I execute $>swift upload con1 OBJ1 -S 1024000 > > I do a force interrupt while send segment 10. > > > > I believe that OBJ1 won't live in con1 , what will happen to the rest > > manifest objects? > > > > Those objects seems still live in con1_segments container. Is there any > > mechanism to audit OBJ1 and delete those manifest objects ? > > > > > > > > -- > > +Hugo Kuo+ > > tonyt...@gmail.com > > +886 935004793 > > > > > > ___ > > Mailing list: https://launchpad.net/~openstack > > Post to : openstack@lists.launchpad.net > > Unsubscribe : https://launchpad.net/~openstack > > More help : https://help.launchpad.net/ListHelp > > > -- +Hugo Kuo+ tonyt...@gmail.com + 886 935004793 ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Swift] Where's Object's metadata located ?
> Q1: Where's the metadata of an object ? It's stored in extended attributes on the filesystem itself. This is reason XFS (or other filesystem supporting extended attributes) is required. > Could I find the value? Sure. You just need some way of a) identifying the object on disk and, b) a means of querying the extended metadata (using for example the python xattrs package). > Q2: What if a large object be interrupted during upload , will the manifest > objects be deleted? Large objects (i.e. those > 5Gb) must be split up client-side and the segments uploaded individually. When all the segments are uploaded the manifest must then be created by the client. What I'm trying to get at is that each segment and even the manifest are completely independent objects. A failure during the upload of any one segment has no impact on other segments or the manifest. Adrian On 16 June 2012 09:53, Kuo Hugo wrote: > Hi folks , > > Q1: > Where's the metadata of an object ? > For example the "X-Object-Manifest". Does it store in inode ? > I did not see the matadata "X-Object=Manifest" in container's DB. > > Could I find the value? > > Q2: > What if a large object be interrupted during upload , will the manifest > objects be deleted? > For example , > OBJ1 :200MB > I execute $>swift upload con1 OBJ1 -S 1024000 > I do a force interrupt while send segment 10. > > I believe that OBJ1 won't live in con1 , what will happen to the rest > manifest objects? > > Those objects seems still live in con1_segments container. Is there any > mechanism to audit OBJ1 and delete those manifest objects ? > > > > -- > +Hugo Kuo+ > tonyt...@gmail.com > +886 935004793 > > > ___ > Mailing list: https://launchpad.net/~openstack > Post to : openstack@lists.launchpad.net > Unsubscribe : https://launchpad.net/~openstack > More help : https://help.launchpad.net/ListHelp > ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] [Swift] Where's Object's metadata located ?
Hi folks , Q1: Where's the metadata of an object ? For example the "X-Object-Manifest". Does it store in inode ? I did not see the matadata "X-Object=Manifest" in container's DB. Could I find the value? Q2: What if a large object be interrupted during upload , will the manifest objects be deleted? For example , OBJ1 :200MB I execute $>swift upload con1 OBJ1 -S 1024000 I do a force interrupt while send segment 10. I believe that OBJ1 won't live in con1 , what will happen to the rest manifest objects? Those objects seems still live in con1_segments container. Is there any mechanism to audit OBJ1 and delete those manifest objects ? -- +Hugo Kuo+ tonyt...@gmail.com + 886 935004793 ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
