It’s kind of fascinating. I imagine that one of the next concepts to come into focus will be “curiosity”. I remember a discussion years ago (15? 18?), I think involving David K., about what the nature of “curiosity” is and what role it plays in learning.
Where the paper talks about supervision to train weights, but eschewing “search” per se as a component of the capability learned, it makes me think of the role of search in the pursuit of inputs, the ultimate worth of which you can’t know at the time of searching. I can imagine (off the cuff) that whatever one wants to mean by “curiosity”, it has some flavor of a non-random search, but one not guided by known criteria, rather by appropriateness to fit existing gaps in (something: confidence? consistency?). This also seems like it should tie into Leslie Valiant’s ideas in Probably Approximately Correct about how to formally conceptualize teaching in relation to learning. I guess Valiant is now considered decades passe, as AI has charged ahead. But the broad outlines of his argument don’t seem like they have become completely superseded. We already have “attention” as a secret sauce with important impacts. I wonder when some shift of architectural paradigm will include a design that we think is a good formalization of the pre-formal gestures toward curiosity. Eric > On Feb 10, 2024, at 8:19 PM, Marcus Daniels <mar...@snoutfarm.com> wrote: > > If one takes results like this -- https://arxiv.org/abs/2402.04494 -- and > then consider what happens with, say, Code Llama, it seems plausible that it > is representing both the breadth and depth of what humans know about large > and complex code bases. It is not clear to me why knowledge can’t be > extended far beyond what the highest-bandwidth humans can learn in a > lifetime. I agree mastery of the idiomatic patterns could constrain > invention, though. For software engineering, the most impressive people to > me are those that can navigate large and complex code bases, often > remembering a lot of the code, but also can discard whole modules at a time > and reimagine them. Managers are suspicious of such people because > managers want to modularize expertise for division of labor. Scrum is in > some sense a way to impede the development of expertise and to deny the need > for it. > > From: Friam <friam-boun...@redfish.com <mailto:friam-boun...@redfish.com>> On > Behalf Of David Eric Smith > Sent: Saturday, February 10, 2024 2:25 AM > To: The Friday Morning Applied Complexity Coffee Group <friam@redfish.com > <mailto:friam@redfish.com>> > Subject: Re: [FRIAM] The problems of interdisciplinary research > > There’s a famous old rant by von Neumann, known at least by those who were > around to hear it, or so I was told by Martin Shubik. > > von Neumann was grumping that “math had become too big; nobody could > understand more than 1/4 of it”. As always with von Neumann, the point of > saying something included an element of self-aggrandizement: von Neumann was > inviting the listener to notice that _he_ was the one who could understand a > quarter of all existing math at the time (whether or not such an absurdity > could be called “true” in any sense). > > I have wondered if this problem marks a qualitative threshold from which to > define a “complex systems” science. The premise would be that all > innovations ultimately occur in individual human heads, triggered somehow. > (And much of the skill of science is to structure your environment of reading > and experience and people to “trigger” you in productive ways, since insight > isn’t something that can be willed into existence). But those ideas need to > be answerable to the fullest scope of whatever is currently understood that > is pertinent. > > The old answer used to be to cram more and more of current knowledge into > single heads as the fuel for their insights, and then to limit to more and > more rarified heads that could hold the most and still come up with > something. > > But at some point, that model no longer works because there is a limit (some > kind of extreme-value distribution, I guess) to what human heads can hold, at > all. > > The project then shifts over into an effort of community design with explicit > concerns that are not reducible to head-packing. How do good insights come > into existence, still limited by heads, but properly responsible to much more > knowledge than the heads do, or even could, contain? > > > I can, of course, shoot down my own way of saying this, immediately. In a > sense, engineers have been doing this for some very very long time. No > “person” knows what is in a 777 aircraft (or for the Europeans, an A380). > Those cases still feel different to me somehow, and like a more standard > expansion of the concept of the assembly line and modularization of tasks > through reliable interfaces (the various ideas behind object design etc.) I > imagine that the interesting problem of idea-finding for complex phenomena > are those that arise when you have modularized as much as you can, and you > have run out of interesting things to add within the modules, because the > things you can’t see transcend them. > > But of course I haven’t “made” anything of this string of words, like a > self-help consultancy or the presidency of any institution. > > Eric > > > > On Feb 9, 2024, at 7:45 PM, Roger Critchlow <r...@elf.org > <mailto:r...@elf.org>> wrote: > > Yeah, it seems like the premise of the cartoon, or maybe Jochen's > interpretation, was that people have limited scopes of application, and the > average scope of application doesn't include interdisciplinary research. But > there are people who have larger scope and have a lot of fun doing > interdisciplinary projects. And if an interdisciplinary group can adapt to > its participant areas of strength, lots of interesting things can happen. > > -- rec -- > > On Fri, Feb 9, 2024 at 3:19 PM Frank Wimberly <wimber...@gmail.com > <mailto:wimber...@gmail.com>> wrote: > I didn't read the article but Carnegie Mellon, where I worked for almost 20 > years, prides itself on the amount of interdisciplinary research accomplished > there.. Herb Simon had appointments in psychology, computer science, > business and public policy, I believe. I was a coauthor of papers in > robotics, public policy, computer science and philosophy. > > On Fri, Feb 9, 2024 at 1:54 PM Jochen Fromm <j...@cas-group.net > <mailto:j...@cas-group.net>> wrote: > Tom Gauld describes most of the problems of interdisciplinary research in a > single image > https://www.newscientist.com/article/2389834-tom-gauld-on-areas-of-expertise/ > > -J. > > -. --- - / ...- .- .-.. .. -.. / -- --- .-. ... . / -.-. --- -.. . > FRIAM Applied Complexity Group listserv > Fridays 9a-12p Friday St. Johns Cafe / Thursdays 9a-12p Zoom > https://bit.ly/virtualfriam > <https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fbit.ly%2fvirtualfriam&c=E,1,0Puf8ZPR6o_m4EcLPFeyqtp89xOLC1YRGDdTEKUSLiOv7k04TG_KSemL6Pk9IIt2FNU9-Pod9znwA38NBaQc8rcYnMa_D_drR6Vw2oKNyf1sOuQlgeoE484KOg,,&typo=1> > to (un)subscribe http://redfish.com/mailman/listinfo/friam_redfish.com > <https://linkprotect.cudasvc.com/url?a=http%3a%2f%2fredfish.com%2fmailman%2flistinfo%2ffriam_redfish.com&c=E,1,76TOEqakoOhXX_lalKacEGa9GGBPx1wfG_o-7boAGsNNBFSM1xrllbT6i-Q0td8CCBIFI2MC0lZ82j04lYnAGX2UT_CZSeOwpdJv7mtk&typo=1> > FRIAM-COMIC http://friam-comic.blogspot.com/ > <https://linkprotect.cudasvc.com/url?a=http%3a%2f%2ffriam-comic.blogspot.com%2f&c=E,1,bEo6w0louxKpJDogqgZDS_p4siVPkg90J0dG_-rJGTiZK4gbwBQkAB2XBXsEbuHQXFgoJnyFY_YKeYkBQo8V-KQacpLi12Gp58S_IyIFlo-m2DhkTXs75aA,&typo=1> > archives: 5/2017 thru present > https://redfish.com/pipermail/friam_redfish.com/ > <https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fredfish.com%2fpipermail%2ffriam_redfish.com%2f&c=E,1,T0mpRAuULI40bTJdAC5zhfPw1IgBMM5Cc_k7LdYWqOgdYGTM-MIWLHM0WDIY4Bs-0JEBrute1ksETud7-ysnH5CMDfv4x0zPpoaBYzReZZQ,&typo=1> > 1/2003 thru 6/2021 http://friam.383.s1.nabble.com/ > > > -- > Frank Wimberly > 140 Calle Ojo Feliz > Santa Fe, NM 87505 > 505 670-9918 > > Research: https://www.researchgate.net/profile/Frank_Wimberly2 > -. --- - / ...- .- .-.. .. -.. / -- --- .-. ... . / -.-. --- -.. . > FRIAM Applied Complexity Group listserv > Fridays 9a-12p Friday St. Johns Cafe / Thursdays 9a-12p Zoom > https://bit.ly/virtualfriam > <https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fbit.ly%2fvirtualfriam&c=E,1,nm8nLWUCiDWvTQZo7VaezJnHw1SkUBsx7oGP8EZT2sNHNPbKU8ACD0YaoPJcdui58vjkPwas_8CzEyB-JD9OoIwUY8QiQfEm3FzPewG2xIWK2-Z2-A,,&typo=1> > to (un)subscribe http://redfish.com/mailman/listinfo/friam_redfish.com > <https://linkprotect.cudasvc.com/url?a=http%3a%2f%2fredfish.com%2fmailman%2flistinfo%2ffriam_redfish.com&c=E,1,nYjZ7o7rSht7G4LXOEBaP-23VQXwlsv7Ox4uR9yFHLnTLsAnwp-2Owj8n5LxM-QFpgrXSHOp8w5bgJmA6gp7u5I2AifrYpWtuPJARkbjxF3j9YAVwoub&typo=1> > FRIAM-COMIC http://friam-comic.blogspot.com/ > <https://linkprotect.cudasvc.com/url?a=http%3a%2f%2ffriam-comic.blogspot.com%2f&c=E,1,OLaZppi63SCuXOH4B5mvvvf-QaCUMv0yEZNfrMTOlbLqVlMkHVi2KMUvMK86q_mx-KDO1jDfyNZm1UZZezOCy2QFc66qrWVW0-FfTlglCJBcDOtY5-IHSbZGsw,,&typo=1> > archives: 5/2017 thru present > https://redfish.com/pipermail/friam_redfish.com/ > <https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fredfish.com%2fpipermail%2ffriam_redfish.com%2f&c=E,1,QFppeetJ33HbN1bMEA8o8Ub_N8q0I-CxRH5HW2u8BcXBoIlx16ZflQo6jZ7583VeFlNgKChAjLFf37xyGqkOAyWpbsEyCV8btjQHHkQcQDUzJGs,&typo=1> > 1/2003 thru 6/2021 http://friam.383.s1.nabble.com/ > -. --- - / ...- .- .-.. .. -.. / -- --- .-. ... . / -.-. --- -.. . > FRIAM Applied Complexity Group listserv > Fridays 9a-12p Friday St. Johns Cafe / Thursdays 9a-12p Zoom > https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fbit.ly%2fvirtualfriam&c=E,1,mYvAAJgz4vxmxboQXnhf2S5ghxO6uGqgGAg2hrgqmRBA-zHjf7Wp9_m9B0d5tENgepFodD1oCUumChgXIdjvCasSuwUUV8ew4-f0o-r5dWE,&typo=1 > to (un)subscribe > https://linkprotect.cudasvc.com/url?a=http%3a%2f%2fredfish.com%2fmailman%2flistinfo%2ffriam_redfish.com&c=E,1,VOgRMqbzbHXQhhQnj0oxqPUn-_oNDZLR320_64XZ8gFXZ--hbCK0GOhBGqaxI6LxXJtjKn7mUZQAyBYq871FtaFzU0J__sHQgieOY22yK2ZmJM37wlhE0llr&typo=1 > FRIAM-COMIC > https://linkprotect.cudasvc.com/url?a=http%3a%2f%2ffriam-comic.blogspot.com%2f&c=E,1,FxWxSLgXCzJwCH4b3PpTYv24JtecA0KbRInBLV33k77u7IaMNE2VULhKCuSn8AJDsdtUPMVukmCuu-jwLXWWbdZQcTRRF6K4-B_b_9Yj9Wo,&typo=1 > archives: 5/2017 thru present > https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fredfish.com%2fpipermail%2ffriam_redfish.com%2f&c=E,1,MIdovpbu4rkq0MYUBES78lX74Jh_jT9ZPIEOw1MjSbWmdrDqinqjphPBKtPiwMJ77SolzF3l0BA1_LVqQ74BKqkCOMMO5YNGptbnHIYjcgW_hcMydLGX4h_MEr4S&typo=1 > 1/2003 thru 6/2021 http://friam.383.s1.nabble.com/ > > -. --- - / ...- .- .-.. .. -.. / -- --- .-. ... . / -.-. --- -.. . > FRIAM Applied Complexity Group listserv > Fridays 9a-12p Friday St. Johns Cafe / Thursdays 9a-12p Zoom > https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fbit.ly%2fvirtualfriam&c=E,1,Sk1DMiTjEqyoEr4n7wpHm3wmn6fCxIbUtCIyIsZDGLH-tW_rwkbF2NJrPxs__H2z-AgCyx6aRK-ZjTbyhP8goT8J8GmyCVCEnRKE4X54ICM,&typo=1 > to (un)subscribe > https://linkprotect.cudasvc.com/url?a=http%3a%2f%2fredfish.com%2fmailman%2flistinfo%2ffriam_redfish.com&c=E,1,cOHQQf5IOW4NeNRtg81Om7E4hHj280QDIx7_JSp5lrieC2m-JWISP4-adgA0HXaDYQ45sW46osXtH4Cj4JCfyVgYlyQawnVsQtUSZRFn8giPtq6Mz7U,&typo=1 > FRIAM-COMIC > https://linkprotect.cudasvc.com/url?a=http%3a%2f%2ffriam-comic.blogspot.com%2f&c=E,1,kTLJyoFKPPssTmgYKRQzaAuIPo3q578urWygco0A80vxva731hSa8sCs8pUmeU8DHRPYPra1lZY62SrPi0wwh5KbFvSXre5UC0cvsWFEYJZvkZjFc9kuI68,&typo=1 > archives: 5/2017 thru present > https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fredfish.com%2fpipermail%2ffriam_redfish.com%2f&c=E,1,OYrm-4KdxxM8O4o9SKFcI0zQ9n3rIwsOT3gP1gv70r_eCkViWDCD1SZ-ZYRH2Ka-v8IX16wnjjJUOS6cSceKz4Yr8_BVp6WRuq-WlUZws4Q,&typo=1 > 1/2003 thru 6/2021 http://friam.383.s1.nabble.com/
-. --- - / ...- .- .-.. .. -.. / -- --- .-. ... . / -.-. --- -.. . FRIAM Applied Complexity Group listserv Fridays 9a-12p Friday St. Johns Cafe / Thursdays 9a-12p Zoom https://bit.ly/virtualfriam to (un)subscribe http://redfish.com/mailman/listinfo/friam_redfish.com FRIAM-COMIC http://friam-comic.blogspot.com/ archives: 5/2017 thru present https://redfish.com/pipermail/friam_redfish.com/ 1/2003 thru 6/2021 http://friam.383.s1.nabble.com/
