Many thanks for the insightful explanation, Steve. T =================================== Tom Johnson - Inst. for Analytic Journalism Santa Fe, NM [email protected] 505-473-9646 ===================================
On Jul 2, 2017 12:33 PM, "Stephen Guerin" <[email protected]> wrote: > Looking into Ofer Dekel's work, the journalist messed up this sentence: > > compress neural networks, the synapses of Machine Learning, down from 32 > bits to, sometimes, a single bit > > > > The team is not compressing neural networks to one-bit. They are > compressing the weights used in a neural network from 32-bits to one-bit. > > Weights (or sometimes called the biases) are proxies for synapses in > biological neural networks. The weights in a neural network are the > activation strengths (or inhibition if negative) on each incoming link to a > node which are multiplied by the outgoing signal strength of the uplink > neighbor. As a number, it can be expressed in 32-bits or as low as 1 bit. > Though at 1-bit weight would not allow for inhibition.The weights are what > get tuned during the machine learning. One can also explore the topology of > the neural network (which nodes are connected to which) during learning and > is the basis for the new craze around Deep Learning. This technique has > been around since the 90s but has now realized its use with the > availability of data. Eg, here's a paper of mine > <http://www.academia.edu/download/5213114/objectgarden.pdf> from '99 > implementing some of research from UT Austin at the time. > > I think it would have been more clear for the journalist to write : > > Ofer Dekel's team is researching methods to reduce the memory and > processing requirements of Neural Networks to run on smaller devices on the > edge of the network closer to the sensors. One method is to reduce the > number of bits necessary describe the weights between nodes in a neural > network down from 32-bits to as little as one-bit. > > -S > _______________________________________________________________________ > [email protected] <[email protected]> > CEO, Simtable http://www.simtable.com > 1600 Lena St #D1, Santa Fe, NM 87505 > office: (505)995-0206 <(505)%20995-0206> mobile: (505)577-5828 > <(505)%20577-5828> > twitter: @simtable > > On Sat, Jul 1, 2017 at 11:42 PM, Tom Johnson <[email protected]> wrote: > >> Friam Friends: >> >> A recent article >> <http://mashable.com/2017/06/29/microsoft-puts-ai-on-a-raspberry-pi/#HKAb_h1pvaqc> >> passed along by George Duncan says: >> >> "Now, Varma's team in India and Microsoft researchers in Redmond, >> Washington, (the entire project is led by lead researcher Ofer Dekel) have >> figured out how to *compress neural networks, the synapses of Machine >> Learning, down from 32 bits to, sometimes, a single bit *and run them on >> a $10 Raspberry Pi >> <http://%20%28the%20entire%20project%20is%20led%20by%20ofer%20dekel%29/>, >> a low-powered, credit-card-sized computer with a handful of ports and no >> screen." >> >> How, or what, can you do with a "single bit."? >> >> TJ >> >> ============================================ >> Tom Johnson >> Institute for Analytic Journalism -- Santa Fe, NM USA >> 505.577.6482 <(505)%20577-6482>(c) >> 505.473.9646 <(505)%20473-9646>(h) >> Society of Professional Journalists <http://www.spj.org> >> *Check out It's The People's Data >> <https://www.facebook.com/pages/Its-The-Peoples-Data/1599854626919671>* >> http://www.jtjohnson.com [email protected] >> ============================================ >> >> ============================================================ >> FRIAM Applied Complexity Group listserv >> Meets Fridays 9a-11:30 at cafe at St. John's College >> to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com >> FRIAM-COMIC http://friam-comic.blogspot.com/ by Dr. Strangelove >> > > > ============================================================ > FRIAM Applied Complexity Group listserv > Meets Fridays 9a-11:30 at cafe at St. John's College > to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com > FRIAM-COMIC http://friam-comic.blogspot.com/ by Dr. Strangelove >
============================================================ FRIAM Applied Complexity Group listserv Meets Fridays 9a-11:30 at cafe at St. John's College to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com FRIAM-COMIC http://friam-comic.blogspot.com/ by Dr. Strangelove
