[GitHub] steffenrost commented on a change in pull request #3418: support cleanup of cloudant/kafka trigger dbs
steffenrost commented on a change in pull request #3418: support cleanup of
cloudant/kafka trigger dbs
URL:
https://github.com/apache/incubator-openwhisk/pull/3418#discussion_r174035748
##
File path:
tests/src/test/scala/whisk/core/database/test/CleanUpWhisksDbSkriptTests.scala
##
@@ -58,26 +58,35 @@ class CleanUpWhisksDbSkriptTests
whisksDbName,
"--dbNameSubjects",
subjectsDbName,
+"--whiskDBType",
+whiskDbType,
"--days",
-"1",
+"7",
"--docsPerRequest",
"1")
val rr = TestUtils.runCmd(0, new File("."), cmd: _*)
-val Seq(marked, deleted, skipped, kept) =
- Seq("marking: ", "deleting: ", "skipping: ", "keeping: ").map {
linePrefix =>
+rr.stdout.lines.foreach {
+ case (line) =>
+println(s"line: $line")
+}
+
+val Seq(marking, marked, deleting, skipping, keeping, statistic) =
+ Seq("marking: ", "marked: ", "deleting: ", "skipping: ", "keeping: ",
"statistic: ").map { linePrefix =>
rr.stdout.lines.collect {
case line if line.startsWith(linePrefix) => line.replace(linePrefix,
"")
}.toList
}
-println(s"marked: $marked")
-println(s"deleted: $deleted")
-println(s"skipped: $skipped")
-println(s"kept:$kept")
+println(s"marked for deletion: $marking")
+println(s"already marked: $marked")
+println(s"deleted: $deleting")
Review comment:
Ok I will remove the code fragment to print out every single line (I used it
only for debugging), so we are then back to the original behavior of the script.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] steffenrost commented on a change in pull request #3418: support cleanup of cloudant/kafka trigger dbs
steffenrost commented on a change in pull request #3418: support cleanup of
cloudant/kafka trigger dbs
URL:
https://github.com/apache/incubator-openwhisk/pull/3418#discussion_r173878169
##
File path: tools/db/cleanUpWhisks.py
##
@@ -95,56 +98,108 @@ def deleteWhisk(dbWhisks, wdoc):
def checkNamespace(dbSubjects, namespace):
while True:
-
allNamespaces = dbSubjects.view('subjects/identities',
startkey=[namespace], endkey=[namespace])
-
if allNamespaces:
return True
else:
return False
+#
+# update last namespace info and print number of kept documents
+#
+def updateLastNamespaceInfo(namespace, lastNamespaceInfo):
+
+if namespace != lastNamespaceInfo[0]:
+if lastNamespaceInfo[0] != None:
+print('keeping: {0} doc(s) for namespace
{1}'.format(lastNamespaceInfo[1], lastNamespaceInfo[0]))
+lastNamespaceInfo = (namespace,0)
+lastNamespaceInfo = (namespace, lastNamespaceInfo[1]+1)
+return lastNamespaceInfo
+
+
+#
+# get delimiters for namespace retrieval by database type
+#
+def getDelimiters(whiskDatabaseType):
+
+if whiskDatabaseType == "whisks":
+# "_id": "whisk.system/utils",
+return None, '/'
+elif whiskDatabaseType == "cloudanttrigger":
+# "_id": ":AHA04676_dev:vision-cloudant-trigger",
Review comment:
@rabbah ah ok, @cbickel and me discussed that point and thought it would be
better to hide the internals and let the caller only specify the type of the
whisk database that needs to be cleaned up, otherwise you would require
knowledge about how the documents look like and what the script will done in
order to obtain the namespace to check. I will discuss with @cbickel tomorrow
about your point.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] steffenrost commented on a change in pull request #3418: support cleanup of cloudant/kafka trigger dbs
steffenrost commented on a change in pull request #3418: support cleanup of
cloudant/kafka trigger dbs
URL:
https://github.com/apache/incubator-openwhisk/pull/3418#discussion_r173802382
##
File path: tools/db/cleanUpWhisks.py
##
@@ -95,56 +98,108 @@ def deleteWhisk(dbWhisks, wdoc):
def checkNamespace(dbSubjects, namespace):
while True:
-
allNamespaces = dbSubjects.view('subjects/identities',
startkey=[namespace], endkey=[namespace])
-
if allNamespaces:
return True
else:
return False
+#
+# update last namespace info and print number of kept documents
+#
+def updateLastNamespaceInfo(namespace, lastNamespaceInfo):
+
+if namespace != lastNamespaceInfo[0]:
+if lastNamespaceInfo[0] != None:
+print('keeping: {0} doc(s) for namespace
{1}'.format(lastNamespaceInfo[1], lastNamespaceInfo[0]))
+lastNamespaceInfo = (namespace,0)
+lastNamespaceInfo = (namespace, lastNamespaceInfo[1]+1)
+return lastNamespaceInfo
+
+
+#
+# get delimiters for namespace retrieval by database type
+#
+def getDelimiters(whiskDatabaseType):
+
+if whiskDatabaseType == "whisks":
+# "_id": "whisk.system/utils",
+return None, '/'
+elif whiskDatabaseType == "cloudanttrigger":
+# "_id": ":AHA04676_dev:vision-cloudant-trigger",
Review comment:
@rabbah didn't get your last point, the database type is passed as parameter
(`--whiskDBType`) and by that the delimiters for the namespace extract from the
document id are determined.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
