[jupyter] Re: Searching over multiple Jupyter notebooks
Ooh, that's neat... but doesn't give a search engine like output? What I really need is a results listing that shows some context for the search hit; I struck on the cell as a convenient proxy for that. --tony On Friday, 11 May 2018 16:04:01 UTC+1, Grant Nestor wrote: > > You can also create a notebook to do this! > > ```py > import glob > > pattern = './**/*.ipynb' > query = 'vdom' > > for filepath in glob.iglob(pattern, recursive=True): > with open(filepath) as file: > s = file.read() > if (s.find(query) > -1): > print(filepath) > ``` > > It gets the job done and it's flexible and you're already using it! > > On Thursday, May 10, 2018 at 10:40:46 AM UTC-7, Tony Hirst wrote: >> >> Thanks for that. I also started dabbling with a simple lunr.js solution - >> initial notes here: >> https://blog.ouseful.info/2018/05/10/initial-sketch-searching-jupyter-notebooks-using-lunr/ >> >> Comments welcome... I need to walk the dog and ponder the actual >> usefulness - or otherwise - of this now. Minimal working demo throws up all >> sorts of issues. COunterpoint of the grep solution is also really useful. A >> third point of comparison would be a sqlite/datasette or >> sqlite/scriptedForm search tool. >> >> --tony >> >> On Wednesday, 9 May 2018 16:42:32 UTC+1, Grant Nestor wrote: >>> >>> A simple solution is to open a terminal in JupyterLab/Jupyter Notebook >>> and run the following: >>> >>> grep --include='*.ipynb' --exclude-dir='.ipynb_checkpoints' -rliw . -e >>> 'search >>> query' >>> >>> This will search your Jupyter server root recursively for files that >>> contain the whole word (case-insensitive) "search query" and only return >>> the file names of matches. >>> >>> More info: >>> https://stackoverflow.com/questions/16956810/how-do-i-find-all-files-containing-specific-text-on-linux >>> >>> On Tuesday, May 8, 2018 at 6:07:02 AM UTC-7, Tony Hirst wrote: Hi I'm working in an edu context, with notebooks being used to deliver interactive teaching materials, and one of the things we know students do is search over reference/resource materials. I was wondering if anyone has looked at simple search solutions for searching over jupyter notebooks, eg by dropping them into a lunr.js index using lunr.py, or adding them to sqlite (in which case, what sort of schema did you use?). In first instance, I was thinking of just indexing the markdown cells in each notebook, with a reference back to the original filepath. (I think effective code search may be a whole other issue.) There are also issues around whether to have views back into a complete notebook, or link to nbconverted html notebooks vs live running notebooks. V first steps in my thinking, just wondered if it's already work in p rogress somewhere? --tony >>> -- You received this message because you are subscribed to the Google Groups "Project Jupyter" group. To unsubscribe from this group and stop receiving emails from it, send an email to jupyter+unsubscr...@googlegroups.com. To post to this group, send email to jupyter@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jupyter/0181120d-c805-4542-a892-76f3966e8bc2%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[jupyter] Re: Searching over multiple Jupyter notebooks
You can also create a notebook to do this! ```py import glob pattern = './**/*.ipynb' query = 'vdom' for filepath in glob.iglob(pattern, recursive=True): with open(filepath) as file: s = file.read() if (s.find(query) > -1): print(filepath) ``` It gets the job done and it's flexible and you're already using it! On Thursday, May 10, 2018 at 10:40:46 AM UTC-7, Tony Hirst wrote: > > Thanks for that. I also started dabbling with a simple lunr.js solution - > initial notes here: > https://blog.ouseful.info/2018/05/10/initial-sketch-searching-jupyter-notebooks-using-lunr/ > > Comments welcome... I need to walk the dog and ponder the actual > usefulness - or otherwise - of this now. Minimal working demo throws up all > sorts of issues. COunterpoint of the grep solution is also really useful. A > third point of comparison would be a sqlite/datasette or > sqlite/scriptedForm search tool. > > --tony > > On Wednesday, 9 May 2018 16:42:32 UTC+1, Grant Nestor wrote: >> >> A simple solution is to open a terminal in JupyterLab/Jupyter Notebook >> and run the following: >> >> grep --include='*.ipynb' --exclude-dir='.ipynb_checkpoints' -rliw . -e >> 'search >> query' >> >> This will search your Jupyter server root recursively for files that >> contain the whole word (case-insensitive) "search query" and only return >> the file names of matches. >> >> More info: >> https://stackoverflow.com/questions/16956810/how-do-i-find-all-files-containing-specific-text-on-linux >> >> On Tuesday, May 8, 2018 at 6:07:02 AM UTC-7, Tony Hirst wrote: >>> >>> Hi >>> >>> I'm working in an edu context, with notebooks being used to deliver >>> interactive >>> teaching materials, and one of the things we know students do is search >>> over reference/resource materials. >>> >>> I was wondering if anyone has looked at simple search solutions for >>> searching >>> over jupyter notebooks, eg by dropping them into a lunr.js index using >>> lunr.py, or adding them to sqlite (in which case, what sort of schema >>> did you use?). >>> >>> In first instance, I was thinking of just indexing the markdown cells >>> in each notebook, with a reference back to the original filepath. (I >>> think effective code search may be a whole other issue.) There are also >>> issues around whether to have views back into a complete notebook, or >>> link to nbconverted html notebooks vs live running notebooks. >>> >>> V first steps in my thinking, just wondered if it's already work in prog >>> ress somewhere? >>> >>> >>> --tony >>> >> -- You received this message because you are subscribed to the Google Groups "Project Jupyter" group. To unsubscribe from this group and stop receiving emails from it, send an email to jupyter+unsubscr...@googlegroups.com. To post to this group, send email to jupyter@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jupyter/fe3c9462-0637-43ea-8cd4-2b0140a66304%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[jupyter] Re: Searching over multiple Jupyter notebooks
Thanks for that. I also started dabbling with a simple lunr.js solution - initial notes here: https://blog.ouseful.info/2018/05/10/initial-sketch-searching-jupyter-notebooks-using-lunr/ Comments welcome... I need to walk the dog and ponder the actual usefulness - or otherwise - of this now. Minimal working demo throws up all sorts of issues. COunterpoint of the grep solution is also really useful. A third point of comparison would be a sqlite/datasette or sqlite/scriptedForm search tool. --tony On Wednesday, 9 May 2018 16:42:32 UTC+1, Grant Nestor wrote: > > A simple solution is to open a terminal in JupyterLab/Jupyter Notebook and > run the following: > > grep --include='*.ipynb' --exclude-dir='.ipynb_checkpoints' -rliw . -e > 'search > query' > > This will search your Jupyter server root recursively for files that > contain the whole word (case-insensitive) "search query" and only return > the file names of matches. > > More info: > https://stackoverflow.com/questions/16956810/how-do-i-find-all-files-containing-specific-text-on-linux > > On Tuesday, May 8, 2018 at 6:07:02 AM UTC-7, Tony Hirst wrote: >> >> Hi >> >> I'm working in an edu context, with notebooks being used to deliver >> interactive >> teaching materials, and one of the things we know students do is search >> over reference/resource materials. >> >> I was wondering if anyone has looked at simple search solutions for >> searching >> over jupyter notebooks, eg by dropping them into a lunr.js index using >> lunr.py, or adding them to sqlite (in which case, what sort of schema >> did you use?). >> >> In first instance, I was thinking of just indexing the markdown cells in >> each notebook, with a reference back to the original filepath. (I think >> effective code search may be a whole other issue.) There are also issues >> around whether to have views back into a complete notebook, or link to >> nbconverted html notebooks vs live running notebooks. >> >> V first steps in my thinking, just wondered if it's already work in progress >> somewhere? >> >> >> --tony >> > -- You received this message because you are subscribed to the Google Groups "Project Jupyter" group. To unsubscribe from this group and stop receiving emails from it, send an email to jupyter+unsubscr...@googlegroups.com. To post to this group, send email to jupyter@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jupyter/7c017cb9-efe8-42ba-b9ae-3fe10c830db5%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[jupyter] Re: Searching over multiple Jupyter notebooks
A simple solution is to open a terminal in JupyterLab/Jupyter Notebook and run the following: grep --include='*.ipynb' --exclude-dir='.ipynb_checkpoints' -rliw . -e 'search query' This will search your Jupyter server root recursively for files that contain the whole word (case-insensitive) "search query" and only return the file names of matches. More info: https://stackoverflow.com/questions/16956810/how-do-i-find-all-files-containing-specific-text-on-linux On Tuesday, May 8, 2018 at 6:07:02 AM UTC-7, Tony Hirst wrote: > > Hi > > I'm working in an edu context, with notebooks being used to deliver > interactive > teaching materials, and one of the things we know students do is search > over reference/resource materials. > > I was wondering if anyone has looked at simple search solutions for searching > over jupyter notebooks, eg by dropping them into a lunr.js index using > lunr.py, > or adding them to sqlite (in which case, what sort of schema did you > use?). > > In first instance, I was thinking of just indexing the markdown cells in > each notebook, with a reference back to the original filepath. (I think > effective code search may be a whole other issue.) There are also issues > around > whether to have views back into a complete notebook, or link to nbconverted > html notebooks vs live running notebooks. > > V first steps in my thinking, just wondered if it's already work in progress > somewhere? > > > --tony > -- You received this message because you are subscribed to the Google Groups "Project Jupyter" group. To unsubscribe from this group and stop receiving emails from it, send an email to jupyter+unsubscr...@googlegroups.com. To post to this group, send email to jupyter@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jupyter/dca3d2e9-5f66-4c56-a3e7-d854fac0e907%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
