On 2/26/13 10:01 AM, Olemis Lang wrote:
On 2/25/13, Jure Zitnik <[email protected]> wrote:
On 2/24/13 7:07 AM, Olemis Lang wrote:
On 2/22/13, Jure Zitnik <[email protected]> wrote:
[....]
Let's say that the 'environment_created' method of the component does
the following:
1. creates table named 'my_table'
2. inserts into 'my_table' data that aggregates/references some data
from system tables ('ticket', 'component',...)
3. creates a file in 'env.path' that holds some environment specific data
4. inserts it's database version into 'system' table
Now, let's discuss the relevant scenarios: (global) environment creation
(install from scratch), product addition and environment upgrade.
There's an important case : upgrade with installed plugins and
preexisting data . maybe covered by the third one . It involves data
migration and a few other corner cases ... but your initial scenarios
are just find to get this sorted out .
I started upgrade scenario discussion in a separate thread last week,
subject '[BEP-0003] Custom (3rd party plugin) table upgrade to
multi-product'.
I'll
describe scenarios for both cases, first for non multi-product aware
component and then for multi-product aware component.
I'll follow this sequence with the purpose in mind of finding a way to
work around the current design limitation of having O(p) plugin tables
.
So I'm basically thinking of having one of the following DB schemas :
1. A single table scoping all plugin resources at product level
2. Two tables : one unchanged for products in global scope ,
the second one for product resources .
jftr, I understand reasoning for trying to find out another way and not
go with the O(p) plugin tables. Please note that the current approach
has been proposed in BEP-0003 a couple of months ago and that all work
done till now was following that design. The option of adding 'product'
column to 3rd party plugin tables (the same way as for system tables)
was assessed when preparing the BEP-0003 database translation proposal.
It was dropped for the reasons stated below.
The proposed 2 options for the database schema are not fundamentally
different. We could go with single table by adding the 'product' column
and scoping the table the same way as we do with the system resource
tables. I believe that's (1) in your list. In both scenarios we run into
the same obstacles as explained below.
Again, IMO that is not doable mainly because we don't know
(semantically) what's in those tables. We don't know the schema, we
don't know what data it could aggregate and/or reference, we don't know
what changes to the schema and data could be performed during the
upgrade etc.
I'll only care about DB stuff because that's actually what really
matters and the cause of this bottleneck .
I'd say that the DB stuff matters, but we can't just ignore other issues.
I'll also ignore MP-aware
plugins / components given the fact that there is no such thing yet
and we will be able to do whatever we consider necessary to make that
work , as opposite to legacy plugins .
+1
1. Global environment creation
-- Non multi-product aware component
In this case, the components
'IEnvironmetSetupParticipant.environment_created' should be executed in
global environment only (as there are no products). Result of this would
be 'my_table' table with aggregated/referenced data from 'null' product
scope (global scope). Note that the translator does not prefix table
names when product is 'null'. File that the component creates would be
stored in the global environment path, the 'system' table would get a
record with 'null' product prefix.
[...]
Looking at this with the previous goals in mind , IMO at this point we
should be doing this (numbers will match scenarios listed above)
1. create plugin table with extra 'product' column for prefix
and perform all other outstanding DDL and DML statements
based on this modified schema .
2. create plugin table with extra 'product' prefix column
How would we be handling indexes, unique keys and other possible
constraint, etc. in any of the above cases? Automatically adding
'product' column to unique constraints? I don't believe there's a
general rule that we could apply in regards to (but not limited to)
constraints...
[...]
2. Product (with 'MYPRODUCT' prefix) addition
-- Non multi-product aware component
Adding a product should invoke component's
'IEnvironmentSetupParticipant.environment_created' method
Ok . We agree up to this point , afaict since the beginning .
+1
from within
'ProductEnvironment('MYPRODUCT')'.
Nope . I disagree here . IMO , this should be happeing immediately
after creating the product either in MP system's CRUD handler , a
dedicated implementation of a product resource listener or a
combination of both .
I understand this, though I don't agree for the reasons I stated above.
In my opinion, and I think Andrej also mentioned this in his reply, from
our perspective what happens within component's
'IEnvironmentSetupParticipant' method is a black box. We can do the SQL
translation through our proxy but I'm strongly against recording SQLs
and doing replays.
For example, how would MP system (or whoever we choose to do the
replays) replay the following really simple SQL, per-product:
SELECT * FROM component WHERE name='Product A component';
assuming that 'Product A component' is only defined in product 'A' ...
This would enable SQL translator in
that scope with the known rules. Result of this would be
'MYPRODUCT_my_table' with aggregated/referenced data from 'MYPRODUCT's
scope. File would be stored in 'MYPRODUCT' product's path. 'system'
table would get a record with 'MYPRODUCT' product prefix.
In both scenarios I mentioned above (1) and (2) we'll need the
translators , that's a fact.
Nevertheless notice that global upgrade would shape the schema for any
product scope as well . So my idea is to shut off DDL statements at
this point . OTOH DML statements will remain translated considering
product prefix as data still needs to be inserted / updated / selected
/ removed .
As much as I would like to, but i really don't think the 'would shape
the schema' would work for reasons stated above (constraints etc.).
I don't understand what you meant with 'shut off DDL statements'? If
nothing else, all DDLs would need to be translated in such a way to take
into account the 'product' column.
[...]
3. Environment upgrade
-- Non multi-product aware component
Following the normal process (in 'trac.env.Environment'), each non
multi-product setup participant should be upgraded (if required) from
all defined product scopes. This would leverage
'IEnvironmentSetupParticipant.environment_needs_upgrade' and
'IEnvironmentSetupParticipant.upgrade_environment', executing those two
from within all product scopes (+global one).
JFTR : this is part of what I referred before as «replay» as I was
actually thinking of an upgrade with products in the DB , preexisting
plugin data and pending upgrades applied upon products in a batch
while still inside the global upgrade loop . IMO this should happen
inside the global upgrade loop .
Upgrade needs to be discussed, I agree, that's why I started a
discussion last week.
[...]
This is how, in my understanding, things should be. I'm not sure there's
really any other way of implementing this ...
We shall see what you all think about my proposed hack . I look
forward to read further comments .
My proposition would still be to go with the O(p) solution. We might
think of making translator a bit more configurable as Andrej suggested.
Cheers,
Jure
