[incubator-sdap-nexus] branch master updated: deletebyquery: Parameter to set the number of rows to fetch from Solr (#217)

[rkk]

This is an automated email from the ASF dual-hosted git repository.

rkk pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-sdap-nexus.git


The following commit(s) were added to refs/heads/master by this push:
     new a85a4bd  deletebyquery: Parameter to set the number of rows to fetch 
from Solr (#217)
a85a4bd is described below

commit a85a4bd8eb408238938f303204dc1f137a176f8c
Author: Riley Kuttruff <72955101+rkuttr...@users.noreply.github.com>
AuthorDate: Tue Feb 14 09:43:11 2023 -0800

    deletebyquery: Parameter to set the number of rows to fetch from Solr (#217)
    
    * Parameter to set the number of rows to fetch from Solr
    
    * Moved changelog entry
    
    * Inhibit cassandra logging
    
    ---------
    
    Co-authored-by: rileykk <rile...@jpl.nasa.gov>
---
 CHANGELOG.md                         |  1 +
 tools/deletebyquery/deletebyquery.py | 11 +++++++++++
 2 files changed, 12 insertions(+)

diff --git a/CHANGELOG.md b/CHANGELOG.md
index 394318f..4ed345f 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -6,6 +6,7 @@ and this project adheres to [Semantic 
Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 ### Added
+- Deletebyquery: Parameter to set the number of rows to fetch from Solr. 
Speeds up time to gather tiles to delete; especially when there is a lot of 
them.
 ### Changed
 ### Deprecated
 ### Removed
diff --git a/tools/deletebyquery/deletebyquery.py 
b/tools/deletebyquery/deletebyquery.py
index 8ed70dd..4fb7bd6 100644
--- a/tools/deletebyquery/deletebyquery.py
+++ b/tools/deletebyquery/deletebyquery.py
@@ -41,6 +41,8 @@ logging.getLogger().setLevel(logging.INFO)
 logging.getLogger().handlers[0].setFormatter(
     logging.Formatter(fmt="%(asctime)s %(levelname)s:%(name)s:  %(message)s", 
datefmt="%Y-%m-%dT%H:%M:%S"))
 
+logging.getLogger('cassandra').setLevel(logging.CRITICAL)
+
 
 def init(args):
     global solr_connection
@@ -89,6 +91,8 @@ def delete_by_query(args):
     else:
         raise RuntimeError("either query or jsonparams is required")
 
+    query.commonparams.rows(args.rows)
+
     if check_query(query):
         logging.info("Collecting tiles ....")
         solr_docs = do_solr_query(query)
@@ -260,6 +264,13 @@ def parse_args():
                         choices=['1', '2', '3', '4', '5'],
                         default='3')
 
+    parser.add_argument('--solr-rows',
+                        help='Number of rows to fetch with each Solr query to 
build the list of tiles to delete',
+                        required=False,
+                        dest='rows',
+                        default=1000,
+                        type=int)
+
     return parser.parse_args()