[GitHub] lucene-solr pull request #455: WIP: SOLR-12638
Github user dsmiley commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/455#discussion_r223561565
--- Diff:
solr/core/src/java/org/apache/solr/update/processor/AtomicUpdateDocumentMerger.java
---
@@ -461,5 +466,33 @@ private static boolean isChildDoc(Object obj) {
}
return objValues.iterator().next() instanceof SolrDocumentBase;
}
+
+ private void removeObj(Collection original, Object toRemove, String
fieldName) {
+if(isChildDoc(toRemove)) {
+ removeChildDoc(original, (SolrInputDocument) toRemove);
+} else {
+ original.remove(getNativeFieldValue(fieldName, toRemove));
+}
+ }
+
+ private static void removeChildDoc(Collection original,
SolrInputDocument docToRemove) {
+for(SolrInputDocument doc: (Collection) original) {
+ if(isDerivedFromDoc(doc, docToRemove)) {
+original.remove(doc);
+return;
+ }
+}
+ }
+
+ private static boolean isDerivedFromDoc(SolrInputDocument fullDoc,
SolrInputDocument subDoc) {
--- End diff --
I see the new docs; still it's unclear without the context of our
conversation. Should "subDoc" be "partialDoc"? How is it that the fullDoc is
"the document to be tested" when the return value is "wether subDoc is a subset
of fullDoc"? Isn't the subject of the test subDoc?
---
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] lucene-solr pull request #455: WIP: SOLR-12638
Github user moshebla commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/455#discussion_r223359889
--- Diff:
solr/core/src/java/org/apache/solr/update/processor/AtomicUpdateDocumentMerger.java
---
@@ -461,5 +466,33 @@ private static boolean isChildDoc(Object obj) {
}
return objValues.iterator().next() instanceof SolrDocumentBase;
}
+
+ private void removeObj(Collection original, Object toRemove, String
fieldName) {
+if(isChildDoc(toRemove)) {
+ removeChildDoc(original, (SolrInputDocument) toRemove);
+} else {
+ original.remove(getNativeFieldValue(fieldName, toRemove));
+}
+ }
+
+ private static void removeChildDoc(Collection original,
SolrInputDocument docToRemove) {
+for(SolrInputDocument doc: (Collection) original) {
+ if(isDerivedFromDoc(doc, docToRemove)) {
+original.remove(doc);
--- End diff --
I was thinking that if doc {"id": "4"} was supplied, then the **whole**
child doc with that id(4) is to be removed.
Or perhaps you have a different scenario in mind, where a field of a a
particular child document is to be removed while the rest of the child document
is left intact?
---
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] lucene-solr pull request #455: WIP: SOLR-12638
Github user moshebla commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/455#discussion_r223355907
--- Diff:
solr/core/src/java/org/apache/solr/update/processor/AtomicUpdateDocumentMerger.java
---
@@ -461,5 +466,33 @@ private static boolean isChildDoc(Object obj) {
}
return objValues.iterator().next() instanceof SolrDocumentBase;
}
+
+ private void removeObj(Collection original, Object toRemove, String
fieldName) {
+if(isChildDoc(toRemove)) {
+ removeChildDoc(original, (SolrInputDocument) toRemove);
+} else {
+ original.remove(getNativeFieldValue(fieldName, toRemove));
+}
+ }
+
+ private static void removeChildDoc(Collection original,
SolrInputDocument docToRemove) {
+for(SolrInputDocument doc: (Collection) original) {
+ if(isDerivedFromDoc(doc, docToRemove)) {
+original.remove(doc);
+return;
+ }
+}
+ }
+
+ private static boolean isDerivedFromDoc(SolrInputDocument fullDoc,
SolrInputDocument subDoc) {
--- End diff --
Exactly,
I will add Javadocs for this.
---
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] lucene-solr pull request #455: WIP: SOLR-12638
Github user moshebla commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/455#discussion_r223355767
--- Diff:
solr/core/src/java/org/apache/solr/update/processor/AtomicUpdateDocumentMerger.java
---
@@ -461,5 +466,33 @@ private static boolean isChildDoc(Object obj) {
}
return objValues.iterator().next() instanceof SolrDocumentBase;
}
+
+ private void removeObj(Collection original, Object toRemove, String
fieldName) {
+if(isChildDoc(toRemove)) {
+ removeChildDoc(original, (SolrInputDocument) toRemove);
+} else {
+ original.remove(getNativeFieldValue(fieldName, toRemove));
+}
+ }
+
+ private static void removeChildDoc(Collection original,
SolrInputDocument docToRemove) {
+for(SolrInputDocument doc: (Collection) original) {
+ if(isDerivedFromDoc(doc, docToRemove)) {
+original.remove(doc);
+return;
+ }
+}
+ }
+
+ private static boolean isDerivedFromDoc(SolrInputDocument fullDoc,
SolrInputDocument subDoc) {
+for(SolrInputField subSif: subDoc) {
+ String fieldName = subSif.getName();
+ if(!fullDoc.containsKey(fieldName)) return false;
+ Collection fieldValues = subDoc.getFieldValues(fieldName);
--- End diff --
fieldValues was supposed to be:
`fullDoc.getFieldValues(fieldName);`
---
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] lucene-solr pull request #455: WIP: SOLR-12638
Github user moshebla commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/455#discussion_r223336215
--- Diff:
solr/core/src/java/org/apache/solr/update/processor/AtomicUpdateDocumentMerger.java
---
@@ -461,5 +466,33 @@ private static boolean isChildDoc(Object obj) {
}
return objValues.iterator().next() instanceof SolrDocumentBase;
}
+
+ private void removeObj(Collection original, Object toRemove, String
fieldName) {
+if(isChildDoc(toRemove)) {
+ removeChildDoc(original, (SolrInputDocument) toRemove);
+} else {
+ original.remove(getNativeFieldValue(fieldName, toRemove));
+}
+ }
+
+ private static void removeChildDoc(Collection original,
SolrInputDocument docToRemove) {
+for(SolrInputDocument doc: (Collection) original) {
+ if(isDerivedFromDoc(doc, docToRemove)) {
+original.remove(doc);
+return;
+ }
+}
+ }
+
+ private static boolean isDerivedFromDoc(SolrInputDocument fullDoc,
SolrInputDocument subDoc) {
+for(SolrInputField subSif: subDoc) {
+ String fieldName = subSif.getName();
+ if(!fullDoc.containsKey(fieldName)) return false;
+ Collection fieldValues = subDoc.getFieldValues(fieldName);
--- End diff --
Oh yes you are correct.
I will get to this and test this ASAP.
---
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] lucene-solr pull request #455: WIP: SOLR-12638
Github user dsmiley commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/455#discussion_r223244019
--- Diff:
solr/core/src/java/org/apache/solr/update/processor/AtomicUpdateDocumentMerger.java
---
@@ -461,5 +466,33 @@ private static boolean isChildDoc(Object obj) {
}
return objValues.iterator().next() instanceof SolrDocumentBase;
}
+
+ private void removeObj(Collection original, Object toRemove, String
fieldName) {
+if(isChildDoc(toRemove)) {
+ removeChildDoc(original, (SolrInputDocument) toRemove);
+} else {
+ original.remove(getNativeFieldValue(fieldName, toRemove));
+}
+ }
+
+ private static void removeChildDoc(Collection original,
SolrInputDocument docToRemove) {
+for(SolrInputDocument doc: (Collection) original) {
+ if(isDerivedFromDoc(doc, docToRemove)) {
+original.remove(doc);
+return;
+ }
+}
+ }
+
+ private static boolean isDerivedFromDoc(SolrInputDocument fullDoc,
SolrInputDocument subDoc) {
--- End diff --
Was does it mean to be "derived" here? Perhaps do you mean that subDoc is
a partial update? Javadocs would help.
---
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] lucene-solr pull request #455: WIP: SOLR-12638
Github user dsmiley commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/455#discussion_r223243871
--- Diff:
solr/core/src/java/org/apache/solr/update/processor/AtomicUpdateDocumentMerger.java
---
@@ -461,5 +466,33 @@ private static boolean isChildDoc(Object obj) {
}
return objValues.iterator().next() instanceof SolrDocumentBase;
}
+
+ private void removeObj(Collection original, Object toRemove, String
fieldName) {
+if(isChildDoc(toRemove)) {
+ removeChildDoc(original, (SolrInputDocument) toRemove);
+} else {
+ original.remove(getNativeFieldValue(fieldName, toRemove));
+}
+ }
+
+ private static void removeChildDoc(Collection original,
SolrInputDocument docToRemove) {
+for(SolrInputDocument doc: (Collection) original) {
+ if(isDerivedFromDoc(doc, docToRemove)) {
+original.remove(doc);
--- End diff --
eh... I'm not sure if it's as simple as `original.remove(doc)` since if it
was, we wouldn't need the method `isDerivedFromDoc`. Assuming we do need that
"derived" predicate method, then we probably need to call remove on an iterator
here.
---
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] lucene-solr pull request #455: WIP: SOLR-12638
Github user dsmiley commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/455#discussion_r223244909
--- Diff:
solr/core/src/test/org/apache/solr/update/processor/AtomicUpdateBlockTest.java
---
@@ -181,6 +182,180 @@ public void testBlockRealTimeGet() throws Exception {
);
}
+ @Test
+ public void testBlockAtomicSet() throws Exception {
+SolrInputDocument doc = sdoc("id", "1",
+"cat_ss", new String[] {"aaa", "ccc"},
+"child1", Collections.singleton(sdoc("id", "2", "cat_ss", "child"))
+);
+json(doc);
+addDoc(adoc(doc), "nested-rtg");
+
+BytesRef rootDocId = new BytesRef("1");
+SolrCore core = h.getCore();
+SolrInputDocument block = RealTimeGetComponent.getInputDocument(core,
rootDocId, true);
+// assert block doc has child docs
+assertTrue(block.containsKey("child1"));
+
+assertJQ(req("q","id:1")
+,"/response/numFound==0"
+);
+
+// commit the changes
+assertU(commit());
+
+SolrInputDocument committedBlock =
RealTimeGetComponent.getInputDocument(core, rootDocId, true);
+BytesRef childDocId = new BytesRef("2");
+// ensure the whole block is returned when resolveBlock is true and id
of a child doc is provided
+assertEquals(committedBlock.toString(),
RealTimeGetComponent.getInputDocument(core, childDocId, true).toString());
+
+assertJQ(req("q","id:1")
+,"/response/numFound==1"
+);
+
+assertJQ(req("qt","/get", "id","1", "fl","id, cat_ss, child1, [child]")
+,"=={\"doc\":{'id':\"1\"" +
+", cat_ss:[\"aaa\",\"ccc\"],
child1:[{\"id\":\"2\",\"cat_ss\":[\"child\"]}]" +
+" }}"
+);
+
+assertU(commit());
+
+assertJQ(req("qt","/get", "id","1", "fl","id, cat_ss, child1, [child]")
+,"=={\"doc\":{'id':\"1\"" +
+", cat_ss:[\"aaa\",\"ccc\"],
child1:[{\"id\":\"2\",\"cat_ss\":[\"child\"]}]" +
+" }}"
+);
+
+doc = sdoc("id", "1",
+"cat_ss", ImmutableMap.of("set", Arrays.asList("aaa", "bbb")),
--- End diff --
Minor: in cases where the JDK has alternatives to Guava, use them. Here,
it's `Collections.singletonMap` I recall.
---
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] lucene-solr pull request #455: WIP: SOLR-12638
Github user dsmiley commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/455#discussion_r223244186
--- Diff:
solr/core/src/java/org/apache/solr/update/processor/AtomicUpdateDocumentMerger.java
---
@@ -461,5 +466,33 @@ private static boolean isChildDoc(Object obj) {
}
return objValues.iterator().next() instanceof SolrDocumentBase;
}
+
+ private void removeObj(Collection original, Object toRemove, String
fieldName) {
+if(isChildDoc(toRemove)) {
+ removeChildDoc(original, (SolrInputDocument) toRemove);
+} else {
+ original.remove(getNativeFieldValue(fieldName, toRemove));
+}
+ }
+
+ private static void removeChildDoc(Collection original,
SolrInputDocument docToRemove) {
+for(SolrInputDocument doc: (Collection) original) {
+ if(isDerivedFromDoc(doc, docToRemove)) {
+original.remove(doc);
+return;
+ }
+}
+ }
+
+ private static boolean isDerivedFromDoc(SolrInputDocument fullDoc,
SolrInputDocument subDoc) {
+for(SolrInputField subSif: subDoc) {
+ String fieldName = subSif.getName();
+ if(!fullDoc.containsKey(fieldName)) return false;
+ Collection fieldValues = subDoc.getFieldValues(fieldName);
--- End diff --
Can't we get this directly off the subSif via subSif.getValues()? And why
would subSif.getValueCount ever be different than fieldValues.size()? Comments
would help.
---
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] lucene-solr pull request #455: WIP: SOLR-12638
Github user moshebla commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/455#discussion_r219034542
--- Diff:
solr/core/src/java/org/apache/solr/update/processor/DistributedUpdateProcessor.java
---
@@ -1184,7 +1196,16 @@ protected boolean versionAdd(AddUpdateCommand cmd)
throws IOException {
// TODO: possibly set checkDeleteByQueries as a flag on the
command?
doLocalAdd(cmd);
-
+
+if(lastKnownVersion != null &&
req.getSchema().isUsableForChildDocs() &&
--- End diff --
This delete ensures the updated block is the latest block, deleting all
previous ones.
---
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] lucene-solr pull request #455: WIP: SOLR-12638
Github user dsmiley commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/455#discussion_r218293921
--- Diff:
solr/core/src/java/org/apache/solr/update/processor/DistributedUpdateProcessor.java
---
@@ -1184,7 +1196,16 @@ protected boolean versionAdd(AddUpdateCommand cmd)
throws IOException {
// TODO: possibly set checkDeleteByQueries as a flag on the
command?
doLocalAdd(cmd);
-
+
+if(lastKnownVersion != null &&
req.getSchema().isUsableForChildDocs() &&
--- End diff --
This may very well be right but can you tell me why you added this delete
here at this line and why the delete is version-dependent?
---
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
