[jira] [Commented] (OAK-6820) Implement support for disabling indexes which are replaced with newer index
[
https://issues.apache.org/jira/browse/OAK-6820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16205417#comment-16205417
]
Chetan Mehrotra commented on OAK-6820:
--
Applied the patch with 1812242
> Implement support for disabling indexes which are replaced with newer index
> ---
>
> Key: OAK-6820
> URL: https://issues.apache.org/jira/browse/OAK-6820
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: indexing
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
> Fix For: 1.8
>
> Attachments: OAK-6820-v1.patch, OAK-6820-v2.patch
>
>
> For upgrade case in many applications older index type is set to {{disabled}}
> when new index is provisioned. If the new index is async then it would take
> some time for reindex and till then any query which used to make use of old
> index would end up traversing the repository
> To avoid such a scenario we should only mark older index as "disabled" only
> if the newer index is reindex.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OAK-6820) Implement support for disabling indexes which are replaced with newer index
[
https://issues.apache.org/jira/browse/OAK-6820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16205035#comment-16205035
]
Vikas Saurabh commented on OAK-6820:
bq. But then we would need to have different index definitions for upgrade and
non upgrade case.
Ack. Missed this point. Agreed that warn can lead to more confusion.
bq. IndexUpdate would set a hidden flag :disableIndexesOnNextCycle if it finds
during reindexing
Yes, this seems better as we would explicitly be tracking the flow.
> Implement support for disabling indexes which are replaced with newer index
> ---
>
> Key: OAK-6820
> URL: https://issues.apache.org/jira/browse/OAK-6820
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: indexing
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
> Fix For: 1.8
>
> Attachments: OAK-6820-v1.patch
>
>
> For upgrade case in many applications older index type is set to {{disabled}}
> when new index is provisioned. If the new index is async then it would take
> some time for reindex and till then any query which used to make use of old
> index would end up traversing the repository
> To avoid such a scenario we should only mark older index as "disabled" only
> if the newer index is reindex.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OAK-6820) Implement support for disabling indexes which are replaced with newer index
[
https://issues.apache.org/jira/browse/OAK-6820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204997#comment-16204997
]
Chetan Mehrotra commented on OAK-6820:
--
bq. while validating existence of superceded index def with declaredNodeTypes
and not finding type after @ in declaredNodeTypes - maybe we should warn
That was my initial thought. But then we would need to have different index
definitions for upgrade and non upgrade case. As for fresh setups those
nodetype entries would anyway not be there
bq. there could be a case where we only update the def to supercede an (async)
index def but not reindex yet - that, afaics, would lead to disabling
superseded defs in next cycle (as reindex flag wouldn've have changed in the
cycle)
Good point. What about we change the logic slightly.
# IndexUpdate would set a hidden flag {{:disableIndexesOnNextCycle}} if it
finds during reindexing
## that current index has {{supersedes}} specified
## And those supersedes refer to indexPaths which have yet not be disabled
# IndexDisabling logic would then only be activated if that flag is present.
And then we do not require managing {{disabledOlderIndexes}}. Post disabling it
would remove that hidden flag
This approach would ensure that indexes are only and only disabled post reindex
as such a hidden flag can only be set by system after due validation
> Implement support for disabling indexes which are replaced with newer index
> ---
>
> Key: OAK-6820
> URL: https://issues.apache.org/jira/browse/OAK-6820
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: indexing
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
> Fix For: 1.8
>
> Attachments: OAK-6820-v1.patch
>
>
> For upgrade case in many applications older index type is set to {{disabled}}
> when new index is provisioned. If the new index is async then it would take
> some time for reindex and till then any query which used to make use of old
> index would end up traversing the repository
> To avoid such a scenario we should only mark older index as "disabled" only
> if the newer index is reindex.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OAK-6820) Implement support for disabling indexes which are replaced with newer index
[
https://issues.apache.org/jira/browse/OAK-6820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204773#comment-16204773
]
Vikas Saurabh commented on OAK-6820:
Patch lgtm. A few points:
* while validating existence of superceded index def with declaredNodeTypes and
_not_ finding type after {{@}} in declaredNodeTypes - maybe we should warn
(yes, one can check the INFO message at the log... but maybe an explicit warn
would be useful too)
* Afaics, just setting {{supersedes}} property on an index def could
potentially disable the indices:
** this diverges a bit from a property on index def taking effect even before
it gets to the hidden index def (although, I guess, that hidden stored index is
only relevant for lucene indices)
** Although, point above is a little too specific to lucene... but, maybe this
feature should at least work only for async indices - sync indices technically
should be able to handle such cases in a single save anyway (without waiting
for reindex before disabling)
** continuing with the first point of hidden def - there could be a case where
we only update the def to supercede an (async) index def but *not* reindex yet
- that, afaics, would lead to disabling superseded defs in next cycle (as
{{reindex}} flag wouldn've have changed in the cycle)
> Implement support for disabling indexes which are replaced with newer index
> ---
>
> Key: OAK-6820
> URL: https://issues.apache.org/jira/browse/OAK-6820
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: indexing
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
> Fix For: 1.8
>
> Attachments: OAK-6820-v1.patch
>
>
> For upgrade case in many applications older index type is set to {{disabled}}
> when new index is provisioned. If the new index is async then it would take
> some time for reindex and till then any query which used to make use of old
> index would end up traversing the repository
> To avoid such a scenario we should only mark older index as "disabled" only
> if the newer index is reindex.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OAK-6820) Implement support for disabling indexes which are replaced with newer index
[
https://issues.apache.org/jira/browse/OAK-6820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16203153#comment-16203153
]
Vikas Saurabh commented on OAK-6820:
Other than what we discussed the other about that IndexUpdate would now need
some sort of understanding of index def, the idea seems fine to me.
Btw, this opens up another question: while sure disabling a prop index is the
quickest way - but, maybe, we should also need to expose some jmx to clean up
such indices - maybe your recursive delete (OAK-6787) could become useful in
this case too? Yes, that' a separate issue, of course.
> Implement support for disabling indexes which are replaced with newer index
> ---
>
> Key: OAK-6820
> URL: https://issues.apache.org/jira/browse/OAK-6820
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: indexing
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
> Fix For: 1.8
>
>
> For upgrade case in many applications older index type is set to {{disabled}}
> when new index is provisioned. If the new index is async then it would take
> some time for reindex and till then any query which used to make use of old
> index would end up traversing the repository
> To avoid such a scenario we should only mark older index as "disabled" only
> if the newer index is reindex.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OAK-6820) Implement support for disabling indexes which are replaced with newer index
[
https://issues.apache.org/jira/browse/OAK-6820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16203107#comment-16203107
]
Chetan Mehrotra commented on OAK-6820:
--
While doing this we would also need to support disabling entries in nodetype
index like removing specific nodetype from
/oak:index/nodetype/@declaringNodeType
> Implement support for disabling indexes which are replaced with newer index
> ---
>
> Key: OAK-6820
> URL: https://issues.apache.org/jira/browse/OAK-6820
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: indexing
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
> Fix For: 1.8
>
>
> For upgrade case in many applications older index type is set to {{disabled}}
> when new index is provisioned. If the new index is async then it would take
> some time for reindex and till then any query which used to make use of old
> index would end up traversing the repository
> To avoid such a scenario we should only mark older index as "disabled" only
> if the newer index is reindex.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OAK-6820) Implement support for disabling indexes which are replaced with newer index
[
https://issues.apache.org/jira/browse/OAK-6820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16201854#comment-16201854
]
Chetan Mehrotra commented on OAK-6820:
--
Discussed this with [~catholicon] and we can do following
# On newer index definition set a mvp property {{supercedes}} which refers to
path of existing index definition which it replaces
{noformat}
/oak:index/assetType
- jcr:primaryType = "oak:QueryIndexDefinition"
- compatVersion = 2
- type = "lucene"
- async = "async"
- supercedes = ['/oak:index/status', '/oak:index/assetType']
{noformat}
# IndexUpdate would check for this property. If present and current index
"reindex" is false then it would check the "type" property of referred index
paths. If they are not set to "disabled" then it would set them to "disabled".
It would also set {{disabledOlderIndexes}} to true. This flag would be used to
avoid this check for later indexing cycles
# If the index replaces yet another index later then it can remove the
{{disabledOlderIndexes}} flag
[~catholicon] [~tmueller] Thoughts?
> Implement support for disabling indexes which are replaced with newer index
> ---
>
> Key: OAK-6820
> URL: https://issues.apache.org/jira/browse/OAK-6820
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: indexing
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
> Fix For: 1.8
>
>
> For upgrade case in many applications older index type is set to {{disabled}}
> when new index is provisioned. If the new index is async then it would take
> some time for reindex and till then any query which used to make use of old
> index would end up traversing the repository
> To avoid such a scenario we should only mark older index as "disabled" only
> if the newer index is reindex.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
