Re: [Opensim-dev] oddities with asset storage

2009-02-24 Thread Dirk Krause 
Sorry, this mail somehow escaped me by scrolling off too fast.

Now that I found it, I'd like to state that a reaping strategy is something 
that definitely will work, but to me is something like the last resort. 

I just imagine all these mails of type:
'Dear Mr. Grid Owner, I hate you because you deleted my Blue Bird Of Happiness 
and my Shiny Sword of Trademark. Yes, I didn't thouch them for over a year, but 
they remind me of the first useless things I ever collected. So be you and your 
first born child please cursed. Thanks for your attention.'

From an IT standpoint I'd like to make sure that you have a reasonable 
database growth, and a reaping strategy would enforce that, so that would be 
ok.

If there'd be another strategy that would make reaping unnecessary, I'd welcome 
that. To me one solution involving different data responsibilities and quotas 
smells like such a solution.

-- Dirk/Bart

-Ursprüngliche Nachricht-
Von: opensim-dev-boun...@lists.berlios.de 
[mailto:opensim-dev-boun...@lists.berlios.de] Im Auftrag von Melanie
Gesendet: Donnerstag, 19. Februar 2009 03:27
An: opensim-dev@lists.berlios.de
Betreff: Re: [Opensim-dev] oddities with asset storage

Hi,

i proposed a reaping strategy that is both type and time based. 
There was no direct response to it, but current development on new 
de-duping cable beach plugins may go a long way to curb asset 
proliferation and the reaper can be developed on that basis. 
Refcounting may be feasible as a positive indication, e.g. to 
indicate something is _definitely_ safe to delete, but a reaper will 
still be required.

Melanie

Buckaroo Mu wrote:
 Melanie wrote:
 This is something i have though about. However, it would not work in 
 OSGrid. Regions may go away, and they may go away permanently. 
 Anything in a prim inventory at that time is refcounted and would 
 not be released. Ever.
   
 In what what in particular would this be worse than the current 
 situation? Yes, some assets would end up staying around forever. 
 However, unlike the current scheme, some... would not.
 So, you'd need a ref list, to purge invalid refs. That is where the 
 inpracticability begins. A redf list for each and every asset, 
 listing either avatar UUID or region UUID?

   
 Eep, no - wasn't considering suggesting this.
 If assets are 51 mio, how long will the reflist table be?

 Melanie

 Buckaroo Mu wrote:
   
 Here's my L$0.02, for what it's worth - why not maintain a 'reference 
 count' in the asset entry?

 Resident A creates a prim, takes it into inventory. Asset is created, 
 inventory item pointing to asset is created, asset-useagecount++. User 
 gives away 15 copies of item, asset-usagecount+=15. Resident A deletes 
 original item, inventory item goes away, asset-usagecount--.

 Resident B rezzes item on sim. If item is no-copy, asset-usagecount--. 
 If it's copy, and Resident B (or Resident C) takes it back into 
 inventory, asset-usagecount++ (they would have two copies of it). 
 Resident B deletes both copies from inventory, asset-usagecount -=2.

 If asset-usagecount == 0, asset gets deleted.

 Tell me why this wouldn't work. I'm sure there's a simple reason, but I 
 can't think of it. All of this is assuming that the asset is no-mod - 
 any change to the asset creates a new UUID, correct? But if the asset is 
 rezzed, then taken back into inventory, it should have the same UUID, 
 thus adding one to the usage count.

 -Buckaroo

 Melanie wrote:
 
 All you described is design behavior.


 Prim items in world are not assets. They are stored exclusively in
 the prims tables of the region.

 Once taken, they become an asset. The name is totally meaningless,
 it reflects whatever was the name at creation. Nothing else. It
 never changes from there on.

 On deleting the inventory item, assets remain. There is no easy or
 practicable way to conclusively say that an asset is trash, because
 we don't know.

 In your test case, your (human) mind could know this for a fact,
 however, the silicon mind of the asset server can't.

 This is because you may have given the object to another user, or
 put a copy into a prim.

 These copies would refer to the same asset, that is what implictily
 shared means.

 So, any asset may, at any time, have any number of links to it, even
 zero.

 Like a website, you don't know who linked to it. If you remove it,
 you leave dead links. Assets are the same. Except, you get Asset
 missing from database.

 So, assetas are NOT 1-to-1 with inventory items, and they _never_
 get deleted.

 They have to be reaped, and there is a big discussion over how to
 best do that, running right now.

 Melanie


 Dirk Krause wrote:
   
   
 Hi,

 I did a little test with a fresh OpenSim installation (yes, thanks for
 the installer!),
 to get a grip on what I learned from Melanie yesterday.

 I wrote a little python script to help me monitor these tables:
   inventoryStore/inventoryItems
   assetStorage/assets
 http://pastebin.com

Re: [Opensim-dev] oddities with asset storage

2009-02-20 Thread Eugen Leitl 
On Thu, Feb 19, 2009 at 09:44:32PM -0500, Frisby, Adam wrote:
 I'm using Project-Voldemort(.com) for the new osgrid server - it can be cross 
 compiled into C# via IKVM and works just fine. It's newish, but it's got some 
 very nice promising features and it's pretty damn fast.

How well does it handle deliberate data corruption (aka malicious nodes)?
 
-- 
Eugen* Leitl a href=http://leitl.org;leitl/a http://leitl.org
__
ICBM: 48.07100, 11.36820 http://www.ativel.com http://postbiota.org
8B29F6BE: 099D 78BA 2FD3 B014 B08A  7779 75B0 2443 8B29 F6BE
___
Opensim-dev mailing list
Opensim-dev@lists.berlios.de
https://lists.berlios.de/mailman/listinfo/opensim-dev

Re: [Opensim-dev] oddities with asset storage

2009-02-20 Thread Frisby, Adam 
It's for use in a trusted environment only. You can't have an asset cluster in 
an untrusted environment without replicating every asset across every host -- 
otherwise someone could setup, run for a while, then deliberately corrupt the 
hard disk. Given enough machines doing this, you could permanently wipe assets.

Adam

 -Original Message-
 From: opensim-dev-boun...@lists.berlios.de [mailto:opensim-dev-
 boun...@lists.berlios.de] On Behalf Of Eugen Leitl
 Sent: Friday, 20 February 2009 1:26 AM
 To: opensim-dev@lists.berlios.de
 Subject: Re: [Opensim-dev] oddities with asset storage

 On Thu, Feb 19, 2009 at 09:44:32PM -0500, Frisby, Adam wrote:
  I'm using Project-Voldemort(.com) for the new osgrid server - it can
 be cross compiled into C# via IKVM and works just fine. It's newish,
 but it's got some very nice promising features and it's pretty damn
 fast.

 How well does it handle deliberate data corruption (aka malicious
 nodes)?

 --
 Eugen* Leitl a href=http://leitl.org;leitl/a http://leitl.org
 __
 ICBM: 48.07100, 11.36820 http://www.ativel.com http://postbiota.org
 8B29F6BE: 099D 78BA 2FD3 B014 B08A  7779 75B0 2443 8B29 F6BE
 ___
 Opensim-dev mailing list
 Opensim-dev@lists.berlios.de
 https://lists.berlios.de/mailman/listinfo/opensim-dev
___
Opensim-dev mailing list
Opensim-dev@lists.berlios.de
https://lists.berlios.de/mailman/listinfo/opensim-dev

Re: [Opensim-dev] oddities with asset storage

2009-02-20 Thread Stefan Andersson 

 

  i thought the reasoning goes also the other way around:
  if someone has a use case where they'd benefit from using Tahoe with 
  OpenSim, they are free to do it
  
  if it doesn't give much benefit for OpenSim otherwise, that doesn't 
  matter, as long as people who want to use it can, without hurting 
  OpenSim (by requiring a lot of / strange, otherwise unbeneficial 
  changes in the core).
 
 +1 As Stefan said, we're a general platform that should allow people to 
 plugin the modules that they want. OpenSim 
 should benefit the people using it, not those people primarily benefiting 
 OpenSim.


I think we have a general confusion here over whether we're discussing the 
technonolgy for inclusion in core, or whether the technology would be a cool 
thing in general.

 

We would probably never include that kind of experimental code in core without 
it being tried out first - that's what the forge is for.

 

But OpenSim as a platform should always strive to make third party 
implementations like this as easy as possible (examples in point, hypergrid, 
Cable Beach) and listen carefully to the suggestions and wishes of plugin 
implementators.

 

  Ok well so be it for OSGrid.
  
  But perhaps some other user, maybe e.g. some intranet somewhere, has it 
  differently.
 
 +1 again


So write it. It's that simple. We promise to be there to help anyone wanting to 
have a go with documentation and general advice.

 
  Feel free, but there is a wealth of good software as GPL, and I sure 
  hope to be able to use those too. Like I'm happy to use Linux to run 
  our company servers, and it being GPL is just fine for me. But I'm also 
  happy that Ogre is BSD so we are free to license our games however we 
  need.
 
 +1 I have no problems with the GPL general, just this particular project is 
 BSD and I think BSD is better here. If 
 other people prefer BSD always than that is their own personal opinion.


Again, that is partly what the forge is for. You can have GPL modules on the 
forge. It's actually a quite good way to sidestep the whole issue.

 
  but can a 3rd party plugin for using them together still 
  exist? I'm sure it can it an at least internally if I don't tell 
  anyone! :p
 
 Yes, I'm guessing this is what Adam meant - we couldn't bundle Tahoe plugins 
 with OpenSim, not that you wouldn't be free 
 to write a third party plugin for using it.


*looks at the forge*

 
  I sure hope this doesn't explose to a huge license discussion, please 
  just point to a better place or reply in private if that would happen.


Again, we are working to get a better structure with regards to what is 
included in the core distribution and what is not. That is an ongoing process, 
but:

 

* Experimental modules not implementing reference client behaviour should be 
tried out on the forge first. This has the added benefit of boosting the API 
thru external change request sources.

* Modules that rely on linking resources under non-compatible licenses should 
be developed and maintained on the forge.

* We are constantly thinking about how to better integrate the forge projects 
and the core distro. That is also an ongoing process.

 

Please, feel free to start hacking away on an customized and prototypical 
IAssetDataPlugin on the forge. I'm looking forward to seeing it!

 

/Stefan

 
___
Opensim-dev mailing list
Opensim-dev@lists.berlios.de
https://lists.berlios.de/mailman/listinfo/opensim-dev

Re: [Opensim-dev] oddities with asset storage

2009-02-19 Thread Dirk Krause 
As soon as you buy it, the asset gets copied to your regions asset repository 
(if it is something that persists while you are not online, like a house maybe) 
and it goes into your personal repository when it's an attachement (like 
clothing).

-Ursprüngliche Nachricht-
Von: opensim-dev-boun...@lists.berlios.de 
[mailto:opensim-dev-boun...@lists.berlios.de] Im Auftrag von Dr Scofield
Gesendet: Donnerstag, 19. Februar 2009 08:46
An: opensim-dev@lists.berlios.de
Betreff: Re: [Opensim-dev] oddities with asset storage

Dirk Krause wrote:
 Glad you asked :-).
 
 I would do a mixture of the following (and admit that I didn't think it all 
 through to the very end).
 
 - introduce grid wide, region wide and personal (user) asset domains
 - introduce quotas for these
 - allow clones ('byref') assets, and copies that go into one of the domains, 
 resp.
 
 I then would expect to have grid wide assets that are 'always on', region 
 wide assets that only are important when the region is connected, and 
 personal assets that are only visible when the user is online (could even be 
 an FTP server behind a cable beach server).
 
 It boils down to: if someone treasures something, she better keeps it in her 
 treasure chest, in her responsibility (and maintenance cost).
 

ok. so i now buy something in your region, take it with me, rezz it on my
region. how would that work and how would that address your concerns? oh, and my
region is rather volatile, it might be there one moment, it might be off-line
the next.

DrS/dirk
-- 
dr dirk husemann  virtual worlds research  ibm zurich research lab
SL: dr scofield  drscofi...@xyzzyxyzzy.net  http://xyzzyxyzzy.net/
RL: h...@zurich.ibm.com - +41 44 724 8573 - http://www.zurich.ibm.com/~hud/
___
Opensim-dev mailing list
Opensim-dev@lists.berlios.de
https://lists.berlios.de/mailman/listinfo/opensim-dev
___
Opensim-dev mailing list
Opensim-dev@lists.berlios.de
https://lists.berlios.de/mailman/listinfo/opensim-dev

Re: [Opensim-dev] oddities with asset storage

2009-02-19 Thread Stefan Andersson 

Initiatives like this is why I'm a hardcore proponent of keeping the services 
encapsulated, indirected, modular and flexible, ie, not tie core to any given 
technology or assumptions about data storage configuration. 

 

Best regards,
Stefan Andersson
Tribal Media AB

 Date: Thu, 19 Feb 2009 10:29:10 +0100
 From: eu...@leitl.org
 To: opensim-dev@lists.berlios.de
 Subject: Re: [Opensim-dev] oddities with asset storage
 
 On Thu, Feb 19, 2009 at 10:24:40AM +0100, Dirk Krause wrote:
  As soon as you buy it, the asset gets copied to your regions asset 
  repository (if it is something that persists while you are not online, like 
  a house maybe) and it goes into your personal repository when it's an 
  attachement (like clothing).
 
 Speaking about asset repository, Tahoe v1.3 has recently been released
 (see below). Are there any show-stoppers against using Tahoe for asset
 storage?
 
 http://allmydata.org/trac/tahoe/browser/relnotes.txt?rev=20090214000500-92b7f-26d07f519c69a3e086ee4599620f6d2c0c9b2fcd
 
 ANNOUNCING allmydata.org Tahoe, the Least-Authority Filesystem, v1.3
 
 We are pleased to announce the release of version 1.3.0 of Tahoe, the
 Least Authority Filesystem.
 
 Tahoe-LAFS is a secure, decentralized, fault-tolerant filesystem. All
 of the source code is available under a choice of two Free Software,
 Open Source licences.
 
 This filesystem is encrypted and distributed over multiple peers in
 such a way it continues to function even when some of the peers are
 unavailable, malfunctioning, or malicious.
 
 Here is the one-page explanation of the security and fault-tolerance
 properties that it offers:
 
 http://allmydata.org/source/tahoe/trunk/docs/about.html
 
 This is the successor to v1.2, which was released July 21, 2008 [1].
 This is a major new release, adding a repairer, an efficient backup
 command, support for large files, an (S)FTP server, and much more.
 
 See the NEWS file [2] and the known_issues.txt file [3] for more
 information.
 
 In addition to the many new features of Tahoe itself, a crop of related
 projects have sprung up, including Tahoe frontends for Windows and
 Macintosh, two front-ends written in JavaScript, a Tahoe plugin for
 duplicity, a Tahoe plugin for TiddlyWiki, a project to create a new
 backup tool, CIFS/SMB integration, an iPhone app, and three incomplete
 Tahoe frontends for FUSE. See Related Projects on the wiki: [4].
 
 
 COMPATIBILITY
 
 The version 1 branch of Tahoe is the basis of the consumer backup
 product from Allmydata, Inc. -- http://allmydata.com .
 
 Tahoe v1.3 is fully compatible with the version 1 branch of Tahoe.
 Files written by v1.3 clients can be read by clients of all versions
 back to v1.0 unless the file is too large -- files greater than about
 12 GiB (depending on the configuration) can't be read by older clients.
 v1.3 clients can read files produced by clients of all versions since
 v1.0. v1.3 servers can serve clients of all versions back to v1.0 and
 v1.3 clients can use servers of all versions back to v1.0 (but can't
 upload large files to them).
 
 This is the fourth release in the version 1 series. We believe that
 this version of Tahoe is stable enough to rely on as a permanent store
 of valuable data. The version 1 branch of Tahoe will be actively
 supported and maintained for the forseeable future, and future versions
 of Tahoe will retain the ability to read files and directories produced
 by Tahoe v1 for the forseeable future.
 
 
 WHAT IS IT GOOD FOR?
 
 With Tahoe, you can distribute your filesystem across a set of
 computers, such that if some of the computers fail or turn out to be
 malicious, the entire filesystem continues to be available, thanks to
 the remaining computers. You can also share your files with other
 users, using a simple and flexible access control scheme.
 
 Because this software is new, we do not categorically recommend it as
 the sole repository of data which is extremely confidential or
 precious. However, we believe that erasure coding, strong encryption,
 Free/Open Source Software and careful engineering make Tahoe safer than
 common alternatives, such as RAID, removable drive, tape, or on-line
 storage or Cloud storage systems.
 
 This software comes with extensive unit tests [5], and there are no
 known security flaws which would compromise confidentiality or data
 integrity. (For all currently known issues please see the
 known_issues.txt file [2].)
 
 This release of Tahoe is suitable for the friendnet use case [6] --
 it is easy to create a filesystem spread over the computers of you and
 your friends so that you can share disk space and files.
 
 
 LICENCE
 
 You may use this package under the GNU General Public License, version
 2 or, at your option, any later version. See the file COPYING.GPL
 [7] for the terms of the GNU General Public License, version 2.
 
 You may use this package under the Transitive Grace Period Public
 Licence, version 1.0. The Transitive Grace Period Public

Re: [Opensim-dev] oddities with asset storage

2009-02-19 Thread Stefan Andersson 

Dirk,

 

I think you are on the right track here - concentrating on one use case at a 
time. That is probably the way to move forward on this issue.

 

BUT do keep in mind that the overshadowing, out-there-right-now, case is the 
'Linden Viewer Compatible Social World'.

 

So whatever we discuss, that case has to be one of the supported ones.

 

That said, optionality and configuration is (as in, should be) our forte.
 
Best regards,
Stefan Andersson
Tribal Media AB



 
 Date: Thu, 19 Feb 2009 13:40:14 +0100
 From: dirk.kra...@pixelpark.com
 To: opensim-dev@lists.berlios.de
 Subject: Re: [Opensim-dev] oddities with asset storage
 
 Ok, then in my naive little world this would be the cloning business case 
 with some kind of quality of service.
 
 Basically you only sell me a link to your variety of assets. Then we have 
 three cases:
 - you keep this in your personal assets store, which might be something like 
 a stand-alone cable beach server. When this server is unavailable, I cannot 
 access your item.
 - you run or rent some 'home/office space' on a particular region; if the 
 region is unavailable, I again cannot access your item.
 - you run your own grid or rent some 'grid space' on a particular grid; if 
 the grid is unavailable, I again cannot access your item. Assuming that grid 
 uptimes are good, this would be the best option in terms of accessibility.
 
 In all three cases you yourself are a customer of either a region or a grid, 
 because you need some level of QOS, which they provide unless you don't want 
 to do this yourself.
 
 If a grid owner is nice she gives you free access to, say, 20MB of personal 
 inventory.
 
 -Ursprüngliche Nachricht-
 Von: opensim-dev-boun...@lists.berlios.de 
 [mailto:opensim-dev-boun...@lists.berlios.de] Im Auftrag von Melanie
 Gesendet: Donnerstag, 19. Februar 2009 12:58
 An: opensim-dev@lists.berlios.de
 Betreff: Re: [Opensim-dev] oddities with asset storage
 
 The issue I have with the concept is one of copyright. I may not 
 grant the right to keep a local copy of an asset, because I may not 
 want it to work on local installations or another grid, I may not 
 want to give up my script source.
 
 Melanie
 
 Dr Scofield wrote:
  Dirk Krause wrote:
  Glad you asked :-).
  
  I would do a mixture of the following (and admit that I didn't think it 
  all through to the very end).
  
  - introduce grid wide, region wide and personal (user) asset domains
  - introduce quotas for these
  - allow clones ('byref') assets, and copies that go into one of the 
  domains, resp.
  
  I then would expect to have grid wide assets that are 'always on', region 
  wide assets that only are important when the region is connected, and 
  personal assets that are only visible when the user is online (could even 
  be an FTP server behind a cable beach server).
  
  It boils down to: if someone treasures something, she better keeps it in 
  her treasure chest, in her responsibility (and maintenance cost).
  
  
  ok. so i now buy something in your region, take it with me, rezz it on my
  region. how would that work and how would that address your concerns? oh, 
  and my
  region is rather volatile, it might be there one moment, it might be 
  off-line
  the next.
  
  DrS/dirk
 ___
 Opensim-dev mailing list
 Opensim-dev@lists.berlios.de
 https://lists.berlios.de/mailman/listinfo/opensim-dev
 ___
 Opensim-dev mailing list
 Opensim-dev@lists.berlios.de
 https://lists.berlios.de/mailman/listinfo/opensim-dev
___
Opensim-dev mailing list
Opensim-dev@lists.berlios.de
https://lists.berlios.de/mailman/listinfo/opensim-dev

Re: [Opensim-dev] oddities with asset storage

2009-02-19 Thread Tommi Laukkanen 
I think somekind of hybrid of different strategies might work well. Maybe
each storage provider could keep the assets in database and use hash to
ensure there is no duplicates and a separate metadata entry which is ref
counted with the binary. Anyhow this would be up to the specific storage
provider implementation. When for one reason or another asset is referenced
by another storage provider then the other asset provider can clone it to
own storage. Rights for cloning can be deduced simply as follows: If you
have right to copy the asset you have right to clone it to your own service
provider. This would also leave distribution strategies etc. decissions for
the specific storage provider. All we need to do is figure out where and how
different storage providers are used. Can region have dedicated storage
provider? Can user have own storage provider? Can grid choose external
storage provider? How do these different storage providers play together in
all use cases?

regards,
Tommi
___
Opensim-dev mailing list
Opensim-dev@lists.berlios.de
https://lists.berlios.de/mailman/listinfo/opensim-dev

Re: [Opensim-dev] oddities with asset storage

2009-02-19 Thread Melanie 
That won't wash.

If i give copy perm just because i need it to be no trans, that 
doesn't mean that I condone off-grid use or personal backup. It has 
to be separated from the perms as we know them.

Most clothing is no mod/no trans. That doesn't mean that the 
clothing creator would want me to be able to download their stuff, 
meaning I would get the raw textures on my disk, to modify and 
reupload as mine...

That that is easy to do with copybot or by getting it from the 
viewer cache is beside the point here, the amount of criminal energy 
required for content theft would be significantly reduced.

Melanie

Tommi Laukkanen wrote:
 I think somekind of hybrid of different strategies might work well. Maybe
 each storage provider could keep the assets in database and use hash to
 ensure there is no duplicates and a separate metadata entry which is ref
 counted with the binary. Anyhow this would be up to the specific storage
 provider implementation. When for one reason or another asset is referenced
 by another storage provider then the other asset provider can clone it to
 own storage. Rights for cloning can be deduced simply as follows: If you
 have right to copy the asset you have right to clone it to your own service
 provider. This would also leave distribution strategies etc. decissions for
 the specific storage provider. All we need to do is figure out where and how
 different storage providers are used. Can region have dedicated storage
 provider? Can user have own storage provider? Can grid choose external
 storage provider? How do these different storage providers play together in
 all use cases?
 
 regards,
 Tommi
 
 
 
 
 
 ___
 Opensim-dev mailing list
 Opensim-dev@lists.berlios.de
 https://lists.berlios.de/mailman/listinfo/opensim-dev
___
Opensim-dev mailing list
Opensim-dev@lists.berlios.de
https://lists.berlios.de/mailman/listinfo/opensim-dev

Re: [Opensim-dev] oddities with asset storage

2009-02-19 Thread Melanie 
No one would sue over a $0.05 shirt. The point of technical copy 
protection is not to block copying. It is to make it require 
criminal energy.
You can't do it accidentally, or because you don't know better, it 
prevents casual copying. You need to invest criminal energy, 
either by downloading software that does it for you, or by writing 
it. That proves intent. Which is what you need, in several 
countries, of you _do_ want to get them slammed for more than just 
the price of the item.
If you can prove intent, the $0.05 shirt can translate into $1000 in 
penalties - and that is what a creator wants.

Melanie


Tommi Laukkanen wrote:
 I have to admit I did not consider this through before posting. Still after
 reflecting on it a bit I am thinking that we might need to consider this
 from practical view point. Usually all copy right enforcing software is
 cracked in no time and open source is especially vulnerable. So I would
 state that if you can copy something say to your inventory or wear it you
 can copy all dependencies to your asset provider. If someone resells
 your assets or breaks your license in some other way you need to sue them
 which is in reality only real protection for any immaterial rights.
 
 Please note that if someone can even observe any asset in a viewer it is
 trivial to write a version of that viewer which will make local copies of
 anything you want. It might not be trivial for end user but cracked versions
 would be soon available in internet. We can not even defend against that as
 our software is open source and modifications are legit. Even those allowing
 for local copies. I don't think legislation in any country forbits you from
 saving a stream or image to your local disk even if it is copyrighted. It
 becomes illegal if you break the license by for example redistributing
 or selling. Instead of starting a fight we can not win we should make a
 clean way to mark licensing and copyrights to all assets and leave it to the
 morale of the users and legal systems.
 
 regards,
 Tommi
 
 
 
 
 
 ___
 Opensim-dev mailing list
 Opensim-dev@lists.berlios.de
 https://lists.berlios.de/mailman/listinfo/opensim-dev
___
Opensim-dev mailing list
Opensim-dev@lists.berlios.de
https://lists.berlios.de/mailman/listinfo/opensim-dev

Re: [Opensim-dev] oddities with asset storage

2009-02-18 Thread Justin Clark-Casey 
Dirk Krause wrote:
 Hi,
 
 I did a little test with a fresh OpenSim installation (yes, thanks for
 the installer!),
 to get a grip on what I learned from Melanie yesterday.
 
 I wrote a little python script to help me monitor these tables:
   inventoryStore/inventoryItems
   assetStorage/assets
 http://pastebin.com/mc9e6574 , be warned: ugly code.
 
 I started OpenSim and logged in a 'Test User' with the SL viewer.
 
 (Just to mention it - first time log in in as test users creates
 4 'blank' entries in assets.)
 
 The inventoryItems table was initially empty.
 
 Now I rezzed a cube and renamed it to 'p1'.
 inventoryStore/inventoryItems was still empty.
 To my surprise no new entry shows up in assetStorage/assets.

Rezzing an object in a region does not involve the creation of any new assets - 
all the data is stored in unserialized 
form in columns in the region database.

 
 Picking up the cube ('take') created an entry named 'p1' in the
 inventory and in the asset table.

The asset you see is the serialized object pointed to by your inventory item 
entry.

 
 Now I renamed the cube in Test Users inventory to 'p2'.
 The name in inventoryStore/inventoryItems changes to 'p2'.
 The entry in assetStorage/assets stays 'p1'. As mentioned on the list
 before,
 the asset name is useless, since the user only sees the name in the
 inventory.

Exactly.

 
 Now I deleted 'p2' in my inventory - 'p2's parentfolderID changes to
 'Trash'.
 Now I emptied the trash - the inventory table is empty again, which is
 fine,
 but here's the big one:
the assetStorage/assets still holds the reference to 'p1'.

Exactly.

 
 Just to make sure I shut down the simulator and opened it again, and it
 was still there.
 
 Now, doesn't that mean, that the database increases over time with no
 hope to find
 these assets that actually should be gone? or is there some magic
 purging that happens,
 and that I missed?

The problem is that you may have given that item to somebody else.  Giving an 
item does not make an asset copy, it just 
makes an inventory item copy (both inventory items still point towards the same 
asset).

So you may delete your item, but we don't know if the asset referenced by that 
item lives on in someone else's inventory 
(or in an object inventory).  So we can't delete the underlying asset.

 
 This would mean that any grid runs into a severe problem over time.

Yep :).  On a standalone one could implement some cleanup scheme which checks 
everything to see if an asset is still 
referenced, and deletes that asset if it is not.

In grid mode this is a much more difficult problem since references are 
scattered across many different regions servers. 
  The situation is even worse if you are running a grid where not all of them 
are guaranteed to be connected.

 
 Best,
   Dirk/Bart
 ___
 Opensim-dev mailing list
 Opensim-dev@lists.berlios.de
 https://lists.berlios.de/mailman/listinfo/opensim-dev
 


-- 
justincc
Justin Clark-Casey
http://justincc.wordpress.com
___
Opensim-dev mailing list
Opensim-dev@lists.berlios.de
https://lists.berlios.de/mailman/listinfo/opensim-dev

Re: [Opensim-dev] oddities with asset storage

2009-02-18 Thread Dirk Krause 
...

 This would mean that any grid runs into a severe problem over time.

 Yep :).  On a standalone one could implement some cleanup scheme
which checks everything to see
 if an asset is still referenced, and deletes that asset if it is not.
 In grid mode this is a much more difficult problem since references
are scattered across many different
 regions servers.  The situation is even worse if you are running a
grid where not all of them are
 guaranteed to be connected.

But isn't that ... horrible? (in lack of a better/worse word.)

As I said yesterday, IMHO there is no real need to think about
optimizations when you have
a serious blocker like this. I would even go so far that this is a major
roadblock for grid based technologies per se. (grid as in Rosedale's
'Happily now, Second Life has been proven to exist. If we disappeared
tomorrow, the grid would be rebuilt by you.')

I take it the bad news is that any proposed solution to this breaks SL
compatibility?

Maybe now would be a good time to take a step away from it.

___
Opensim-dev mailing list
Opensim-dev@lists.berlios.de
https://lists.berlios.de/mailman/listinfo/opensim-dev

Re: [Opensim-dev] oddities with asset storage

2009-02-18 Thread Stefan Andersson 

Um. Having immutable assets do bring a number of optimization shortcuts. I'm 
not saying it's all good - merely that it's not all bad.


The OpenSim solution, as always, is to address each case by itself, and to make 
divergence optional.

 

So, maybe region map textures should be overwriteable. Maybe some scripts 
should be overwriteable as well, maybe only on some type of scripts. Maybe we 
should have a table enumerating what assetIds now point to newer assetIds.

 

Maybe the real problem is that we are talking about 'assets' and 'textures' 
instead of 'prim face textures', 'sculptie maps', 'photos', 'region map 
images', 'script sources', 'sounds', 'prims in inventory', 'notecards', 
'animations' - my point being that we can probably implement various 
configurable strategies for each of them, linden viewer or no linden viewer.


Best regards,
Stefan Andersson
Tribal Media AB


 Date: Wed, 18 Feb 2009 16:37:12 +0100
 From: dirk.kra...@pixelpark.com
 To: opensim-dev@lists.berlios.de
 Subject: Re: [Opensim-dev] oddities with asset storage
 
 ...
 
  This would mean that any grid runs into a severe problem over time.
 
  Yep :). On a standalone one could implement some cleanup scheme
 which checks everything to see
  if an asset is still referenced, and deletes that asset if it is not.
  In grid mode this is a much more difficult problem since references
 are scattered across many different
  regions servers. The situation is even worse if you are running a
 grid where not all of them are
  guaranteed to be connected.
 
 But isn't that ... horrible? (in lack of a better/worse word.)
 
 As I said yesterday, IMHO there is no real need to think about
 optimizations when you have
 a serious blocker like this. I would even go so far that this is a major
 roadblock for grid based technologies per se. (grid as in Rosedale's
 'Happily now, Second Life has been proven to exist. If we disappeared
 tomorrow, the grid would be rebuilt by you.')
 
 I take it the bad news is that any proposed solution to this breaks SL
 compatibility?
 
 Maybe now would be a good time to take a step away from it.
 
 ___
 Opensim-dev mailing list
 Opensim-dev@lists.berlios.de
 https://lists.berlios.de/mailman/listinfo/opensim-dev

___
Opensim-dev mailing list
Opensim-dev@lists.berlios.de
https://lists.berlios.de/mailman/listinfo/opensim-dev

Re: [Opensim-dev] oddities with asset storage

2009-02-18 Thread Dr Scofield 
Dirk Krause wrote:
 ...
[...]
 
 But isn't that ... horrible? (in lack of a better/worse word.)
 
 As I said yesterday, IMHO there is no real need to think about
 optimizations when you have
 a serious blocker like this. I would even go so far that this is a major
 roadblock for grid based technologies per se. (grid as in Rosedale's
 'Happily now, Second Life has been proven to exist. If we disappeared
 tomorrow, the grid would be rebuilt by you.')
 
 I take it the bad news is that any proposed solution to this breaks SL
 compatibility?
 
 Maybe now would be a good time to take a step away from it.

so, what do you supposed should be done? ride OpenSim on web route 404? lots of
dangling references?

i supposed on a standalone system you could do ref counting or bidirectional
refs/links --- that however is not a very scalable solution for a grid with
sporadically connected grid components.

one avatar's garbage is another avatar's treasure...

DrS/dirk


-- 
dr dirk husemann  virtual worlds research  ibm zurich research lab
SL: dr scofield  drscofi...@xyzzyxyzzy.net  http://xyzzyxyzzy.net/
RL: h...@zurich.ibm.com - +41 44 724 8573 - http://www.zurich.ibm.com/~hud/
___
Opensim-dev mailing list
Opensim-dev@lists.berlios.de
https://lists.berlios.de/mailman/listinfo/opensim-dev

Re: [Opensim-dev] oddities with asset storage

2009-02-18 Thread Melanie 
We can't step away from SL compatibility until there is a
full-featured, viable viewer, preferably not based on Linden Labs
code. Until then, the viewer's asset caching mechanism make that
impossible.

Melanie


Dirk Krause wrote:
 ...
 
 This would mean that any grid runs into a severe problem over time.
 
 Yep :).  On a standalone one could implement some cleanup scheme
 which checks everything to see
 if an asset is still referenced, and deletes that asset if it is not.
 In grid mode this is a much more difficult problem since references
 are scattered across many different
 regions servers.  The situation is even worse if you are running a
 grid where not all of them are
 guaranteed to be connected.
 
 But isn't that ... horrible? (in lack of a better/worse word.)
 
 As I said yesterday, IMHO there is no real need to think about
 optimizations when you have
 a serious blocker like this. I would even go so far that this is a major
 roadblock for grid based technologies per se. (grid as in Rosedale's
 'Happily now, Second Life has been proven to exist. If we disappeared
 tomorrow, the grid would be rebuilt by you.')
 
 I take it the bad news is that any proposed solution to this breaks SL
 compatibility?
 
 Maybe now would be a good time to take a step away from it.
 
 ___
 Opensim-dev mailing list
 Opensim-dev@lists.berlios.de
 https://lists.berlios.de/mailman/listinfo/opensim-dev
 
 

___
Opensim-dev mailing list
Opensim-dev@lists.berlios.de
https://lists.berlios.de/mailman/listinfo/opensim-dev

Re: [Opensim-dev] oddities with asset storage

2009-02-18 Thread Dirk Krause 
There could be business modell attached to it.
Lets say you sell only the 'right to use it for a given time' to the user, then 
you would have only one set of assets with multiple inventory pointers from 
your 2000 customers. Once you delete/disable it, no one can use it anymore. 
Once you update it, all 2000 have a newer version of it.  The other model would 
be the 'I buy it so I get a real copy of it'. The asset will be copied (in my 
scenario to my local inventory domain, so I 'really' own it). If you delete 
yours, mine is still there. If you update it, I need to obtain a newer copy. 
There could even be a concept that something is there only once (maybe art); 
you create it, give it a away (for a good price) and you don't have it anymore. 
The asset vanishes from your inventory domain.

Regarding the SL compatibility - maybe it doesn't need to be broken. IIRC isn't 
there this CAPS mechanism that already proxies assets? Couldn't there be some 
kind of intelligence behind it that collects and distributes assets from the 
different domains?

-Ursprüngliche Nachricht-
Von: opensim-dev-boun...@lists.berlios.de 
[mailto:opensim-dev-boun...@lists.berlios.de] Im Auftrag von Melanie
Gesendet: Mittwoch, 18. Februar 2009 19:57
An: opensim-dev@lists.berlios.de
Betreff: Re: [Opensim-dev] oddities with asset storage

Making a copy is the greater evil. With implicitly shared assets, 
only content creators create assets. With asset copying, each 
sale/give creates assets.
Take SL:

I make a clothing item. I have to make 18 uploads (creating 18 
assets) to finally use 2 of the uploaded textures. I have also 
created 36 wearable assets through this.

Then i put that item for sale. 2000 users buy it.

With implicitly shared assets, assets consumed: 18 texture, 36 wearable.

With asset copying, assets consumed: 4001 texture, 4003 wearable

See, the point of diminishing returns for copying is very close.

Melanie


John Ward wrote:
 Justin Clark-Casey wrote:
 The problem is that you may have given that item to somebody else.
 Giving an item does not make an asset copy, it just makes an
 inventory item copy (both inventory items still point towards the
 same asset).
 
 So you may delete your item, but we don't know if the asset
 referenced by that item lives on in someone else's inventory (or in
 an object inventory).  So we can't delete the underlying asset.
 
 Why not make an asset copy when one makes an inventory copy?  Then 
 delete the asset when deleted from inventory.  Is each user having their 
 own copy of many things a bigger problem?  I guess this doesn't address 
 one having out of ban knowledge of an assets UUID and expecting it to be 
 there.  Also, I accept that I may be missing some fundamental knowledge 
 of how things work.  Please be gentle :-)
 
 John.
 ___
 Opensim-dev mailing list
 Opensim-dev@lists.berlios.de
 https://lists.berlios.de/mailman/listinfo/opensim-dev
 
 
___
Opensim-dev mailing list
Opensim-dev@lists.berlios.de
https://lists.berlios.de/mailman/listinfo/opensim-dev
___
Opensim-dev mailing list
Opensim-dev@lists.berlios.de
https://lists.berlios.de/mailman/listinfo/opensim-dev

Re: [Opensim-dev] oddities with asset storage

2009-02-18 Thread Stefan Andersson 

I suggest we talk more about complementary and parallell scenarios and 
strategies, and less about either/or - this is a platform, and various 
implemetations will prioritize differently.

Best regards,
Stefan Andersson
Tribal Media AB



 
 Date: Wed, 18 Feb 2009 20:02:38 +0100
 From: dirk.kra...@pixelpark.com
 To: opensim-dev@lists.berlios.de
 Subject: Re: [Opensim-dev] oddities with asset storage
 
 There could be business modell attached to it.
 Lets say you sell only the 'right to use it for a given time' to the user, 
 then you would have only one set of assets with multiple inventory pointers 
 from your 2000 customers. Once you delete/disable it, no one can use it 
 anymore. Once you update it, all 2000 have a newer version of it. The other 
 model would be the 'I buy it so I get a real copy of it'. The asset will be 
 copied (in my scenario to my local inventory domain, so I 'really' own it). 
 If you delete yours, mine is still there. If you update it, I need to obtain 
 a newer copy. There could even be a concept that something is there only once 
 (maybe art); you create it, give it a away (for a good price) and you don't 
 have it anymore. The asset vanishes from your inventory domain.
 
 Regarding the SL compatibility - maybe it doesn't need to be broken. IIRC 
 isn't there this CAPS mechanism that already proxies assets? Couldn't there 
 be some kind of intelligence behind it that collects and distributes assets 
 from the different domains?
 
 -Ursprüngliche Nachricht-
 Von: opensim-dev-boun...@lists.berlios.de 
 [mailto:opensim-dev-boun...@lists.berlios.de] Im Auftrag von Melanie
 Gesendet: Mittwoch, 18. Februar 2009 19:57
 An: opensim-dev@lists.berlios.de
 Betreff: Re: [Opensim-dev] oddities with asset storage
 
 Making a copy is the greater evil. With implicitly shared assets, 
 only content creators create assets. With asset copying, each 
 sale/give creates assets.
 Take SL:
 
 I make a clothing item. I have to make 18 uploads (creating 18 
 assets) to finally use 2 of the uploaded textures. I have also 
 created 36 wearable assets through this.
 
 Then i put that item for sale. 2000 users buy it.
 
 With implicitly shared assets, assets consumed: 18 texture, 36 wearable.
 
 With asset copying, assets consumed: 4001 texture, 4003 wearable
 
 See, the point of diminishing returns for copying is very close.
 
 Melanie
 
 
 John Ward wrote:
  Justin Clark-Casey wrote:
  The problem is that you may have given that item to somebody else.
  Giving an item does not make an asset copy, it just makes an
  inventory item copy (both inventory items still point towards the
  same asset).
  
  So you may delete your item, but we don't know if the asset
  referenced by that item lives on in someone else's inventory (or in
  an object inventory). So we can't delete the underlying asset.
  
  Why not make an asset copy when one makes an inventory copy? Then 
  delete the asset when deleted from inventory. Is each user having their 
  own copy of many things a bigger problem? I guess this doesn't address 
  one having out of ban knowledge of an assets UUID and expecting it to be 
  there. Also, I accept that I may be missing some fundamental knowledge 
  of how things work. Please be gentle :-)
  
  John.
  ___
  Opensim-dev mailing list
  Opensim-dev@lists.berlios.de
  https://lists.berlios.de/mailman/listinfo/opensim-dev
  
  
 ___
 Opensim-dev mailing list
 Opensim-dev@lists.berlios.de
 https://lists.berlios.de/mailman/listinfo/opensim-dev
 ___
 Opensim-dev mailing list
 Opensim-dev@lists.berlios.de
 https://lists.berlios.de/mailman/listinfo/opensim-dev
___
Opensim-dev mailing list
Opensim-dev@lists.berlios.de
https://lists.berlios.de/mailman/listinfo/opensim-dev

Re: [Opensim-dev] oddities with asset storage

2009-02-18 Thread Dirk Krause 
Ok, granted.  But isn't this a bit chicken/egg here? Possible solution 
scenarios should define the requirements of the viewer, which probably won't 
write itself.  Esp. when new viewers will be based on openmv somebody should 
tell John et al. :-).

-Ursprüngliche Nachricht-
Von: opensim-dev-boun...@lists.berlios.de 
[mailto:opensim-dev-boun...@lists.berlios.de] Im Auftrag von Melanie
Gesendet: Mittwoch, 18. Februar 2009 20:42
An: opensim-dev@lists.berlios.de
Betreff: Re: [Opensim-dev] oddities with asset storage

In the viewer, the following are true:

- The asset ID attached to an inventory item may not change
- The item ID of an inventory item may not change
- an asset's content may not change.

So, with this client, it's moot.

We are exploring other concepts and will be implementing them as and 
when other clients become usable.

Melanie


Dirk Krause wrote:
 There could be business modell attached to it.
 Lets say you sell only the 'right to use it for a given time' to the user, 
 then you would have only one set of assets with multiple inventory pointers 
 from your 2000 customers. Once you delete/disable it, no one can use it 
 anymore. Once you update it, all 2000 have a newer version of it.  The other 
 model would be the 'I buy it so I get a real copy of it'. The asset will be 
 copied (in my scenario to my local inventory domain, so I 'really' own it). 
 If you delete yours, mine is still there. If you update it, I need to obtain 
 a newer copy. There could even be a concept that something is there only once 
 (maybe art); you create it, give it a away (for a good price) and you don't 
 have it anymore. The asset vanishes from your inventory domain.
 
 Regarding the SL compatibility - maybe it doesn't need to be broken. IIRC 
 isn't there this CAPS mechanism that already proxies assets? Couldn't there 
 be some kind of intelligence behind it that collects and distributes assets 
 from the different domains?

 
 -Ursprüngliche Nachricht-
 Von: opensim-dev-boun...@lists.berlios.de 
 [mailto:opensim-dev-boun...@lists.berlios.de] Im Auftrag von Melanie
 Gesendet: Mittwoch, 18. Februar 2009 19:57
 An: opensim-dev@lists.berlios.de
 Betreff: Re: [Opensim-dev] oddities with asset storage
 
 Making a copy is the greater evil. With implicitly shared assets, 
 only content creators create assets. With asset copying, each 
 sale/give creates assets.
 Take SL:
 
 I make a clothing item. I have to make 18 uploads (creating 18 
 assets) to finally use 2 of the uploaded textures. I have also 
 created 36 wearable assets through this.
 
 Then i put that item for sale. 2000 users buy it.
 
 With implicitly shared assets, assets consumed: 18 texture, 36 wearable.
 
 With asset copying, assets consumed: 4001 texture, 4003 wearable
 
 See, the point of diminishing returns for copying is very close.
 
 Melanie
 
 
 John Ward wrote:
 Justin Clark-Casey wrote:
 The problem is that you may have given that item to somebody else.
 Giving an item does not make an asset copy, it just makes an
 inventory item copy (both inventory items still point towards the
 same asset).
 
 So you may delete your item, but we don't know if the asset
 referenced by that item lives on in someone else's inventory (or in
 an object inventory).  So we can't delete the underlying asset.
 
 Why not make an asset copy when one makes an inventory copy?  Then 
 delete the asset when deleted from inventory.  Is each user having their 
 own copy of many things a bigger problem?  I guess this doesn't address 
 one having out of ban knowledge of an assets UUID and expecting it to be 
 there.  Also, I accept that I may be missing some fundamental knowledge 
 of how things work.  Please be gentle :-)
 
 John.
 ___
 Opensim-dev mailing list
 Opensim-dev@lists.berlios.de
 https://lists.berlios.de/mailman/listinfo/opensim-dev
 
 
 ___
 Opensim-dev mailing list
 Opensim-dev@lists.berlios.de
 https://lists.berlios.de/mailman/listinfo/opensim-dev
 ___
 Opensim-dev mailing list
 Opensim-dev@lists.berlios.de
 https://lists.berlios.de/mailman/listinfo/opensim-dev
 
 

___
Opensim-dev mailing list
Opensim-dev@lists.berlios.de
https://lists.berlios.de/mailman/listinfo/opensim-dev
___
Opensim-dev mailing list
Opensim-dev@lists.berlios.de
https://lists.berlios.de/mailman/listinfo/opensim-dev

Re: [Opensim-dev] oddities with asset storage

2009-02-18 Thread Dirk Krause 
ok, very cool, thanks for the info.

I take it you esp refer to IdealistViewer, openviewer and the Rex-NG viewer?

-Ursprüngliche Nachricht-
Von: opensim-dev-boun...@lists.berlios.de 
[mailto:opensim-dev-boun...@lists.berlios.de] Im Auftrag von Melanie
Gesendet: Mittwoch, 18. Februar 2009 21:21
An: opensim-dev@lists.berlios.de
Betreff: Re: [Opensim-dev] oddities with asset storage

There are several viewer being developed already and their authors 
are aware of requirements and responsive to different needs.

Mainly, any new viewer will be able to accommodate changes quickly, 
unlike the LL viewer. So I see no need for a drawn out 
standardization discussion. This project is in too early a phase for 
this.

Melanie

Dirk Krause wrote:
 Ok, granted.  But isn't this a bit chicken/egg here? Possible solution 
 scenarios should define the requirements of the viewer, which probably won't 
 write itself.  Esp. when new viewers will be based on openmv somebody should 
 tell John et al. :-).
 
 -Ursprüngliche Nachricht-
 Von: opensim-dev-boun...@lists.berlios.de 
 [mailto:opensim-dev-boun...@lists.berlios.de] Im Auftrag von Melanie
 Gesendet: Mittwoch, 18. Februar 2009 20:42
 An: opensim-dev@lists.berlios.de
 Betreff: Re: [Opensim-dev] oddities with asset storage
 
 In the viewer, the following are true:
 
 - The asset ID attached to an inventory item may not change
 - The item ID of an inventory item may not change
 - an asset's content may not change.
 
 So, with this client, it's moot.
 
 We are exploring other concepts and will be implementing them as and 
 when other clients become usable.
 
 Melanie
 
 
 Dirk Krause wrote:
 There could be business modell attached to it.
 Lets say you sell only the 'right to use it for a given time' to the user, 
 then you would have only one set of assets with multiple inventory pointers 
 from your 2000 customers. Once you delete/disable it, no one can use it 
 anymore. Once you update it, all 2000 have a newer version of it.  The other 
 model would be the 'I buy it so I get a real copy of it'. The asset will be 
 copied (in my scenario to my local inventory domain, so I 'really' own it). 
 If you delete yours, mine is still there. If you update it, I need to obtain 
 a newer copy. There could even be a concept that something is there only 
 once (maybe art); you create it, give it a away (for a good price) and you 
 don't have it anymore. The asset vanishes from your inventory domain.
 
 Regarding the SL compatibility - maybe it doesn't need to be broken. IIRC 
 isn't there this CAPS mechanism that already proxies assets? Couldn't there 
 be some kind of intelligence behind it that collects and distributes assets 
 from the different domains?
 
 
 -Ursprüngliche Nachricht-
 Von: opensim-dev-boun...@lists.berlios.de 
 [mailto:opensim-dev-boun...@lists.berlios.de] Im Auftrag von Melanie
 Gesendet: Mittwoch, 18. Februar 2009 19:57
 An: opensim-dev@lists.berlios.de
 Betreff: Re: [Opensim-dev] oddities with asset storage
 
 Making a copy is the greater evil. With implicitly shared assets, 
 only content creators create assets. With asset copying, each 
 sale/give creates assets.
 Take SL:
 
 I make a clothing item. I have to make 18 uploads (creating 18 
 assets) to finally use 2 of the uploaded textures. I have also 
 created 36 wearable assets through this.
 
 Then i put that item for sale. 2000 users buy it.
 
 With implicitly shared assets, assets consumed: 18 texture, 36 wearable.
 
 With asset copying, assets consumed: 4001 texture, 4003 wearable
 
 See, the point of diminishing returns for copying is very close.
 
 Melanie
 
 
 John Ward wrote:
 Justin Clark-Casey wrote:
 The problem is that you may have given that item to somebody else.
 Giving an item does not make an asset copy, it just makes an
 inventory item copy (both inventory items still point towards the
 same asset).
 
 So you may delete your item, but we don't know if the asset
 referenced by that item lives on in someone else's inventory (or in
 an object inventory).  So we can't delete the underlying asset.
 
 Why not make an asset copy when one makes an inventory copy?  Then 
 delete the asset when deleted from inventory.  Is each user having their 
 own copy of many things a bigger problem?  I guess this doesn't address 
 one having out of ban knowledge of an assets UUID and expecting it to be 
 there.  Also, I accept that I may be missing some fundamental knowledge 
 of how things work.  Please be gentle :-)
 
 John.
 ___
 Opensim-dev mailing list
 Opensim-dev@lists.berlios.de
 https://lists.berlios.de/mailman/listinfo/opensim-dev
 
 
 ___
 Opensim-dev mailing list
 Opensim-dev@lists.berlios.de
 https://lists.berlios.de/mailman/listinfo/opensim-dev
 ___
 Opensim-dev mailing list
 Opensim-dev@lists.berlios.de
 https://lists.berlios.de/mailman

Re: [Opensim-dev] oddities with asset storage

2009-02-18 Thread Dirk Krause 
I hope I am not too notorious by stating:
- no 'big corp' IT manager will accept a solution with 'uncontrollable growth 
in asset storage' as Tommi correctly put it. Period.
- I learned through Melanie that these issue are well known and already 
addressed.
- I also learned that potential viewer developers are aware of these issues and 
have it on their list which is great.
- I also agree with Stefan that there should be alternative scenarios possible.


Von: opensim-dev-boun...@lists.berlios.de 
[mailto:opensim-dev-boun...@lists.berlios.de] Im Auftrag von Charles Krinke
Gesendet: Mittwoch, 18. Februar 2009 21:52
An: opensim-dev@lists.berlios.de
Betreff: Re: [Opensim-dev] oddities with asset storage

My goal in starting this whole discussion in the first place was two fold.

Fold 1: Get us considering how to evolve OpenSim so that assets database 
currently containing 1.5million entries and consuming 50GBytes to support 
10,000 users does not continue to grow without bound at the current 4GByte/week 
rate if possible. I see other grid deployments facing a similar issue in their 
future and some evolution at this point may help us all.

Fold 2: I believe there are a few changes such as the last terrain image 
store that we can change in the near term, and I am hoping we can define a 
'reaping' strategy for grids that lets them tame the exponential growth of the 
'assets' table in the MySQL database defined by OpenSim as 2009 continues.

Charles


From: Melanie mela...@t-data.com
To: opensim-dev@lists.berlios.de
Sent: Wednesday, February 18, 2009 12:20:42 PM
Subject: Re: [Opensim-dev] oddities with asset storage

There are several viewer being developed already and their authors 
are aware of requirements and responsive to different needs.

Mainly, any new viewer will be able to accommodate changes quickly, 
unlike the LL viewer. So I see no need for a drawn out 
standardization discussion. This project is in too early a phase for 
this.

Melanie

Dirk Krause wrote:
 Ok, granted.  But isn't this a bit chicken/egg here? Possible solution 
 scenarios should define the requirements of the viewer, which probably won't 
 write itself.  Esp. when new viewers will be based on openmv somebody should 
 tell John et al. :-).
 
 -Ursprüngliche Nachricht-
 Von: opensim-dev-boun...@lists.berlios.de 
 [mailto:opensim-dev-boun...@lists.berlios.de] Im Auftrag von Melanie
 Gesendet: Mittwoch, 18. Februar 2009 20:42
 An: opensim-dev@lists.berlios.de
 Betreff: Re: [Opensim-dev] oddities with asset storage
 
 In the viewer, the following are true:
 
 - The asset ID attached to an inventory item may not change
 - The item ID of an inventory item may not change
 - an asset's content may not change.
 
 So, with this client, it's moot.
 
 We are exploring other concepts and will be implementing them as and 
 when other clients become usable.
 
 Melanie
 
 
 Dirk Krause wrote:
 There could be business modell attached to it.
 Lets say you sell only the 'right to use it for a given time' to the user, 
 then you would have only one set of assets with multiple inventory pointers 
 from your 2000 customers. Once you delete/disable it, no one can use it 
 anymore. Once you update it, all 2000 have a newer version of it.  The other 
 model would be the 'I buy it so I get a real copy of it'. The asset will be 
 copied (in my scenario to my local inventory domain, so I 'really' own it). 
 If you delete yours, mine is still there. If you update it, I need to obtain 
 a newer copy. There could even be a concept that something is there only 
 once (maybe art); you create it, give it a away (for a good price) and you 
 don't have it anymore. The asset vanishes from your inventory domain.
 
 Regarding the SL compatibility - maybe it doesn't need to be broken. IIRC 
 isn't there this CAPS mechanism that already proxies assets? Couldn't there 
 be some kind of intelligence behind it that collects and distributes assets 
 from the different domains?
 
 
 -Ursprüngliche Nachricht-
 Von: opensim-dev-boun...@lists.berlios.de 
 [mailto:opensim-dev-boun...@lists.berlios.de] Im Auftrag von Melanie
 Gesendet: Mittwoch, 18. Februar 2009 19:57
 An: opensim-dev@lists.berlios.de
 Betreff: Re: [Opensim-dev] oddities with asset storage
 
 Making a copy is the greater evil. With implicitly shared assets, 
 only content creators create assets. With asset copying, each 
 sale/give creates assets.
 Take SL:
 
 I make a clothing item. I have to make 18 uploads (creating 18 
 assets) to finally use 2 of the uploaded textures. I have also 
 created 36 wearable assets through this.
 
 Then i put that item for sale. 2000 users buy it.
 
 With implicitly shared assets, assets consumed: 18 texture, 36 wearable.
 
 With asset copying, assets consumed: 4001 texture, 4003 wearable
 
 See, the point of diminishing returns for copying is very close.
 
 Melanie
 
 
 John Ward wrote:
 Justin Clark-Casey wrote:
 The problem

Re: [Opensim-dev] oddities with asset storage

2009-02-18 Thread Ideia Boa 

- I also agree with you



Dirk Krause wrote:

I hope I am not too notorious by stating:
- no 'big corp' IT manager will accept a solution with 'uncontrollable growth 
in asset storage' as Tommi correctly put it. Period.
- I learned through Melanie that these issue are well known and already 
addressed.
- I also learned that potential viewer developers are aware of these issues and 
have it on their list which is great.
- I also agree with Stefan that there should be alternative scenarios possible.


Von: opensim-dev-boun...@lists.berlios.de 
[mailto:opensim-dev-boun...@lists.berlios.de] Im Auftrag von Charles Krinke
Gesendet: Mittwoch, 18. Februar 2009 21:52
An: opensim-dev@lists.berlios.de
Betreff: Re: [Opensim-dev] oddities with asset storage

My goal in starting this whole discussion in the first place was two fold.

Fold 1: Get us considering how to evolve OpenSim so that assets database 
currently containing 1.5million entries and consuming 50GBytes to support 
10,000 users does not continue to grow without bound at the current 4GByte/week 
rate if possible. I see other grid deployments facing a similar issue in their 
future and some evolution at this point may help us all.

Fold 2: I believe there are a few changes such as the last terrain image 
store that we can change in the near term, and I am hoping we can define a 'reaping' 
strategy for grids that lets them tame the exponential growth of the 'assets' table in 
the MySQL database defined by OpenSim as 2009 continues.

Charles


From: Melanie mela...@t-data.com
To: opensim-dev@lists.berlios.de
Sent: Wednesday, February 18, 2009 12:20:42 PM
Subject: Re: [Opensim-dev] oddities with asset storage

There are several viewer being developed already and their authors 
are aware of requirements and responsive to different needs.


Mainly, any new viewer will be able to accommodate changes quickly, 
unlike the LL viewer. So I see no need for a drawn out 
standardization discussion. This project is in too early a phase for 
this.


Melanie

Dirk Krause wrote:
  

Ok, granted.  But isn't this a bit chicken/egg here? Possible solution 
scenarios should define the requirements of the viewer, which probably won't 
write itself.  Esp. when new viewers will be based on openmv somebody should 
tell John et al. :-).

-Ursprüngliche Nachricht-
Von: opensim-dev-boun...@lists.berlios.de 
[mailto:opensim-dev-boun...@lists.berlios.de] Im Auftrag von Melanie
Gesendet: Mittwoch, 18. Februar 2009 20:42
An: opensim-dev@lists.berlios.de
Betreff: Re: [Opensim-dev] oddities with asset storage

In the viewer, the following are true:

- The asset ID attached to an inventory item may not change
- The item ID of an inventory item may not change
- an asset's content may not change.

So, with this client, it's moot.

We are exploring other concepts and will be implementing them as and 
when other clients become usable.


Melanie


Dirk Krause wrote:


There could be business modell attached to it.
Lets say you sell only the 'right to use it for a given time' to the user, then 
you would have only one set of assets with multiple inventory pointers from 
your 2000 customers. Once you delete/disable it, no one can use it anymore. 
Once you update it, all 2000 have a newer version of it.  The other model would 
be the 'I buy it so I get a real copy of it'. The asset will be copied (in my 
scenario to my local inventory domain, so I 'really' own it). If you delete 
yours, mine is still there. If you update it, I need to obtain a newer copy. 
There could even be a concept that something is there only once (maybe art); 
you create it, give it a away (for a good price) and you don't have it anymore. 
The asset vanishes from your inventory domain.

Regarding the SL compatibility - maybe it doesn't need to be broken. IIRC isn't 
there this CAPS mechanism that already proxies assets? Couldn't there be some 
kind of intelligence behind it that collects and distributes assets from the 
different domains?
  
-Ursprüngliche Nachricht-

Von: opensim-dev-boun...@lists.berlios.de 
[mailto:opensim-dev-boun...@lists.berlios.de] Im Auftrag von Melanie
Gesendet: Mittwoch, 18. Februar 2009 19:57
An: opensim-dev@lists.berlios.de
Betreff: Re: [Opensim-dev] oddities with asset storage

Making a copy is the greater evil. With implicitly shared assets, 
only content creators create assets. With asset copying, each 
sale/give creates assets.

Take SL:

I make a clothing item. I have to make 18 uploads (creating 18 
assets) to finally use 2 of the uploaded textures. I have also 
created 36 wearable assets through this.


Then i put that item for sale. 2000 users buy it.

With implicitly shared assets, assets consumed: 18 texture, 36 wearable.

With asset copying, assets consumed: 4001 texture, 4003 wearable

See, the point of diminishing returns for copying is very close.

Melanie


John Ward wrote:
  

Justin Clark-Casey wrote