Perhaps you're looking for https://grep.app/ ? It does regex search vs github and was recently acquired by Vercel. It was written by a friend of mine.
On Wed, Jan 8, 2025 at 9:44 AM anon anon <anonimoussech...@gmail.com> wrote: > Markus: I probably misunderstood your remark. > > Could it be possible to use a git clone protocol plugin please? > > Le mer. 8 janv. 2025 à 15:41, anon anon <anonimoussech...@gmail.com> a > écrit : > > > David: > > > > I also would like to ensure I clarified correctly. > > > > I absolutely need to index source code to my personal search engine to > run > > a regex in solr. I want to look for vulnerabilities with the regex. > > > > COuld you provide the steps for a such configuration of nutch and > > eventually solr please? > > > > Best regards. > > > > Le mer. 8 janv. 2025 à 15:25, anon anon <anonimoussech...@gmail.com> a > > écrit : > > > >> Hello David, > >> > >> I need a git "clone" indexer to index an as huge as possible database of > >> repo to make cyber security research for my job. > >> > >> Hello Markus, > >> > >> I am open to any proposition. > >> > >> I did not found in the doc how to make a git clone only of a repo url > >> from the crawler indexer config regex. I also see in the source code > there > >> https://github.com/apache/nutch/tree/master/src/plugin that the > protocol > >> supported are present there. I doubt I could add my own custom protocol > in > >> config. I hope I am wrong. If you are sure I could glone a repo in nucth > >> config directly, could you tell me how please? > >> > >> If really you think I need to fork the repo, I can do it as well. > >> > >> Best regards. > >> > >> Le mar. 7 janv. 2025 à 16:01, Markus Jelsma <markus.jel...@openindex.io > > > >> a écrit : > >> > >>> Hi, > >>> > >>> Nutch is, just as Solr, highly customizable using all sorts of plugins. > >>> Forking it is not recommended. If you happen to come across behaviour > in > >>> one of its tools that is not configurable, it can be made configurable. > >>> > >>> Regards, > >>> Markus > >>> > >>> Op di 7 jan 2025 om 16:52 schreef David Smiley <dsmi...@apache.org>: > >>> > >>> > Forking anything is a burden on you to maintain your fork. You > didn't > >>> say > >>> > *why* you want to fork something instead of simply use something. > You > >>> > mentioned adding features but search engine platforms like Solr are > >>> > designed to be highly pluggable/extensible without forking. It's a > >>> > platform not a product. > >>> > > >>> > On Sun, Jan 5, 2025 at 6:36 PM anon <anonimoussech...@gmail.com> > >>> wrote: > >>> > > >>> > > Hello people!! > >>> > > > >>> > > I was going to fork sourcegraph because I was looking for a search > >>> > > engine specific to code source such as github and gitlab with the > >>> > > possibility to index decompiled file offline. then I read this > >>> copyright > >>> > > > >>> > > > >>> > > >>> > https://github.com/sourcegraph/sourcegraph-public-snapshot/blob/main/LICENSE.enterprise > >>> > > < > >>> > > > >>> > > >>> > https://github.com/sourcegraph/sourcegraph-public-snapshot/blob/main/LICENSE.enterprise > >>> > > > >>> > > > >>> > > it seems to be *more than* proprietary. Then I just found > >>> opensearch. It > >>> > > seems modular. I might fork it to: > >>> > > 1- index only source code from github/gitlab and from local to my > >>> > instance > >>> > > 2- use regex and codeql queries in the search client. > >>> > > > >>> > > Opensearch seems good but not modular enough. > >>> > > > >>> > > > >>> > > I think, solr the best choice for me. I will complete with a fork > on > >>> > nutch. > >>> > > > >>> > > I think a Nutch fork would absolutely complete what I am looking > for: > >>> > > > >>> > > - it is free software > >>> > > > >>> > > - it is modular on many protocol (not git yet), and solr compatible > >>> > > > >>> > > I suggest that I fork nutch to add a plugin there > >>> > > https://github.com/apache/nutch/tree/master/src/plugin under a new > >>> > > folder protocol-file and why not let people fork it. > >>> > > > >>> > > Is it a good idea? > >>> > > > >>> > > Best regards. > >>> > > > >>> > > >>> > >> > -- http://www.needhamsoftware.com (work) https://a.co/d/b2sZLD9 (my fantasy fiction book)
