pre-existing example at
https://github.com/ggerganov/llama.cpp/blob/b24c3049d96557c24782e4d32feaae65f47277af/examples/train-text-from-scratch/train-text-from-scratch.cpp#L3041
looks like most setup code still manually inlined for dev tweaking
On 6/18/23, Karl Semich <0xl...@gmail.com> wrote:
> i actually had trouble creating the model and building ggml. if i had
i mean comstructing an empty basic transformer architecture here, not
an actual model of anything.
> ignored file storage i would possibly have skipped most of the
> trouble. i
i actually had trouble creating the model and building ggml. if i had
ignored file storage i would possibly have skipped most of the
trouble. i was thinking of linking to ctransformers which is made for
python but builds a library with load and eval functions for popular
transformers
middle
On 6/18/23, Karl Semich <0xl...@gmail.com> wrote:
> ggml provides an opt function that performs training inside it based
> on the number of iterations set in the parameters. this is why the
> example code takes so many minutes to complete, it’s running the opt
> pass 10k times on a single core. at
ggml provides an opt function that performs training inside it based
on the number of iterations set in the parameters. this is why the
example code takes so many minutes to complete, it’s running the opt
pass 10k times on a single core. at a glance it looks like the
performance stats are near
On 6/18/23, Karl Semich <0xl...@gmail.com> wrote:
> here’s the code for the bayesian one:
> https://github.com/gmum/few-shot-hypernets-public/tree/master/methods/hypernets
>
> i’m wondering if there’s more private than public research here dunno
>
> anyway it seems like these papers kind of say
here’s the code for the bayesian one:
https://github.com/gmum/few-shot-hypernets-public/tree/master/methods/hypernets
i’m wondering if there’s more private than public research here dunno
anyway it seems like these papers kind of say the encoding of weights
is a hyperparameter that hasn’t been
this seems helpful since transformers are so normal, it makes a
transformer that makes other transformers:
https://github.com/google-research/google-research/tree/master/hypertransformer
other papers have newer things for example generating a kernel-based
bayesian model that combines information
bumped into this while scrubbing briefly for metalearning information
https://arxiv.org/abs/2301.07628
We introduce the concept of "universal password model" -- a password
model that, once pre-trained, can automatically change its guessing
strategy based on the target system. To achieve this,
i’m considering https://arxiv.org/pdf/2303.07502.pdf which is
purportedly a 2023 meta-learning review. i suspect my idea of the
topic is a little off the mark.
it’s not immediately apparent where to find documentation on training
in ggml but here’s the optimizer test code which performs a single
optimization pass which might be sufficient if it actually works when
nobody seems to be using it:
it looks like something extra is needed at the start or it won’t
succeed without a lot more data collection. a quick idea would be
direct controls that let the user direct the limb before it learns to
infer their intent.
i think it would make sense to try this with ggml, which seems to be
where
— -
ok so i saw a realworld person with handmade feed attached to
amputations right below the knees and i got a rockawesome idea for a
puzzle: free homebrew robotic prosthetic limbs that respond to user
intention accurately.
i think i’ve solved big chunks of the puzzle in outline form. the core
——-
“i need a bus ticket to get to the fair!”
the poor tulip was sad. it wanted to ride to the fair, but nobody was
selling bus tickets to tulips.
it out on its robot legs and walked back and forth
once upon a time a pore dead individual named Poded was infected with
zombieism in their grave
i tried to message jenn about my idea of mind control being the impact
of dissociated urgencies on decision making and how this is a studied
and purportedly curable thing but the messenger is not loading
maybe we could model traditional zombieism as a virus-like disease
that is capable of reanimating foesh and building behavior trends in
reanimated nervous systems
i imagine it’s a scenario a lot of people have thought about, but of
course i identify with brain-eating zombies in multiple ways, and
consider their similarity to a mind control borg as artistically
noncoincidental (re disease patterns, widespread unwise behaviors …)
one of the major problems
——
something i spent some time enjoying thinking about while crazy was
the idea of doing conflict mediation with a brain-eating zombie. i
guess it basically revolved around a shared need for nourishment that
our bodies accepted and was enjoyable to eat, and basically the zombie
ends up eating
breathing instructor: “do you have any rhythmic behaviors that feel
satisfying and energizing to you?”
breathing instructor: “have you ever heard of breathing, dude?”
pause
breathing instructor: “sorry, what i mean is, how do you give oxygen
to your cells in outer space?”
space lifeform: “i extract oxygen from my food when i digest it. i
function on less than most do.”
discussion between breathing instructor and artificial biological
lifeform engineered for life in outer space
boss instructs screaming orchestra
stands in middle
various diverse folk are ready to scream at him to the beat of his baton
message lost
every now and then
the rain falls on my head
and rains peace through my hair
and coolness over my nose
once upon a time oh hmm not once upon a time
boss was running a simulation of bosses
one of his fake bosses refused to act bosslike
fake boss 3: “i don’t want to take over the world! i want to be humble
and respectful.”
fake boss 3 crossed their arms and pouted.
boss looked at fake boss 3.
lost message, something like: the key is to get the rounding error
within the tilt bounds to compensate for timing mistakes pressing the
paddle button and wear on the paddle spring
second lost idea
then, boss was ignoring tilt bounds and just picking up pinball
machine and rolling ball into
the key is to get the rounding error within the tilt bounds to
compensate for your timing mistakes pressing the paddle button and the
wear that develops on the paddle spring
boss plays pinball
the pinball game has a physics and engineering feature where when the
ball strikes a paddle time stops inside the machine and the player is
prompted to solve for the trajectory of the ball after the impact
extended message loss in network issue
this one was lost and isretyped:
boss: “look for whores”
simulated text adventure: “i don’t understand “for whores” to look at
it. you are standing by a well house. what do you do?”
simulated text adventure
welc9me to collossal cave
you are standing by a well house
what do you do
>
boss sets a country of datacenters to simulate a text adventure for him
the datacenters fill warehouses that have giant smokestacks that belch
black smoke
riverse run through them and the water comes out half steam
[they domt want to look down because they dont want to dive so they
keep heading up]
—-
suicidal ground diver is stuck at the top of a mountain
janitor: “now, i know this meeting is about energy infrastructure, but
there is a more pressing issue.”
boss sings a stanza from an opera
janitor: “the problem you see with mr boss here, everybody has been
developing these same problems.”
investor: “some anonymous individuals were able to press
boss lays in a meeting
his limbs jerk around
sometimes he speaks
a visiting dignitary walks by the office. boss had an appointment with
them. the whole mass jumps a little and begins moving toward the
dignitary but most of the parts bump into whatever is in their way in
that direction and get snagged again or more
parts of boss’s brain collect on his worktable and form a lobe here, a
lobe there. parts move between the lobes as if trying to fit
themselves into a puzzle unsuccessfully
some parts slither under the others and make for the door. they pile
up, grab the knob, and open it
a cigar floats in the air unsmoked
boss descends from his office ceiling as a cloud of pixels and
triangles stuck through rips in spacetime. they get snagged on things
they go by and have trouble coming together into something bosslike
boss plays holoretromud
he enters holoretromud and is surrounded by a giant minecraft-like
world of giant voxels
aww man minecraft already made something ridiculously low resolution and modern
yeah it was just going to be vr minecraft with mud dungeon theme
i guess that’s how far i got
looks like return specifications for void funcs are optional
looks like exported symbols need be be titlecased
i tried to put a relative path straight into an import statement but
go complained this was not supported
i redid it with ascii art from banner package by 0716
maybe take a break and write something self-designed
maybe i can find an ascii art package or something and use it.
i’ll wipe my gohello folder and try to write code of my own
https://go.dev/doc/tutorial/create-module
go mod edit —replace example.com/greetings=./greetings #0706
go mod tidy
go run . #0707
slow parts, a significant slowdown is posting updates but these could
also help ground maybe
noting the intent is to build skill via repetition
it’s taking a long time for `go modtidy` to autodetect a new import
and download it because i am on a phone proxy
oh there it goes
go run . # 06:58
the output message is poignant
https://go.dev/doc/tutorial/getting-started
go mod init github.com/xloem/gohello # 06:50
vim hello.go
go run . # 06:51
holy frack i wrote a hello world program in two minutes.
https://go.dev/learn/
i have to decide whether to learn from the docs or a walkthru
i guess i’ll do the walkthru but usually you’d do the docs (so you
learn all the features rather than leaving some out)
ok i found a place with go installed
let’s jtry to speedrun maybe part of a tutorial?
thought:
maybe it could be helpful to find a tutorial but skim it super fast,
skipping most if
a biggest challenge for me could be just setting up a compiler
i wanted to add cloning to gothub, but go confuses me
when considering learning a new language, it’s also valuable to
consider how hard it would be in the future. usual a good software
developer is language agnostic, such that even new languages are easy
to work in.
53 matches
Mail list logo
