just to clarify here,
my understanding is that is a non-moderated community and any list-wide
moderation is non-forthright and appropriately called censorship.
if the posts here become scalpel-like targeted mind control made by
powerful ai slavery systems to turn us all into hamburgers, that's
i'm not a cryptographer, but is sounds like this research might leave out
hybrid approaches where part of the algorithm is handled in a quantum way
and the rest is brute forced as usual?
regarding the idea for saving state, that could work here. basically you
take a fancy text generation model and finetune it to produce its own
embeddings by feeding it one token at a time instead of a document, each
time feeding back its generated state as embeddings. it then is possibly
bound by
Rebased for PR.
https://github.com/AminRezaei0x443/memory-efficient-attention/pull/4
contains excess comments demonstrating craziness.
commit ab6170cedec07a6d7554916c859d36329f1a4125 (HEAD ->
sparse-jax-masks, origin/sparse-jax-masks)
Author: xloem <0xl...@gmail.com>
Date: Wed Feb 2 10:16:24
Appts today.
Below includes implementation of the mask and bias features as the
owner requested. Latest at top.
I still have two changes to make:
- modify if conditions i added to handle all cases, including unworkable ones
- fix bug when chunk size is not a factor of total size. existing code
is
double queue:
- verify email
- download the list of usa person data to devices i want to stop
using, see if that can influence me
Bumping the more accurate thread title after replying to the less accurate one.
When the homeless camps between the highways and railroads in a city
were "cleaned up", the entire areas were clearcut of vegetation. Many
were protected wetlands. People's tents and sleeping bags were
physically slashed and confiscated. The people this happens to don't
have avenues for legal
The points Punk-Stasi raises are common anarchist values.
It's notable that in the usa the police usually do not physically
murder people (although it happens occasionally on a regular basis).
The actions of government and enforcement do cause a lot of indirect
death that people fight. but most
A reply some time ago linked to an old post regarding a "coderpunks"
list but it looked like it was simply a brief mention of an idea, not
an actual list?
But of course there are scads of dev lists out there.
Karl's comments:
The original public focus on directed energy weapons was made after
the american Targeted Individual community contacted their government
with their information, experience, and the supporting documentation
they had collected of being victims of human experimentation since the
For a little thoroughness here's a link to
https://github.com/openai/human-eval which is just code for some old
2021 openai paper, and a quote from its readme:
> This program exists to run untrusted model-generated code. Users are strongly
> encouraged not to do so outside of a robust security
Also this thread subject is rude and painful.
They link to information on their public datasets at
https://github.com/deepmind/code_contests .
It doesn't appear that the model[s] is/are public yet, so it's not
quite that newsworthy yet. People are quite aware there is private
code generation e.g. at openai.com .
There's a 5 month old open
AminRezaei commented on my PR and made some requests I only just reviewed.
I'm guessing that's what's important is resolving top bug below.
Expecting then it to be reasonably easy to meet the requests.
The implementation choice I made, of a general callback() function
provides for users to hack
for casual readers unfamiliar with immediate situation,
i did contact local police. i asked for help sorting out expereinces
to report them to police. they referred me to therapist. therapist
does not like talkinga bout anything to do with police ;p says other
strange things, and behavior of
it's very hard to come clean about murder because the person we killed
was a slug. a small animal. sometimes we consider killing a real
human, to make automated-gaslighting stop.
karl was a murder suspect (once? still?) and was thinking that maybe
this explained some of the "mind control" he could have experienced:
other people responding grossly to his subtle behaviors.
if a murder detective were working with facebook, to monitor what
people respond to for example, that
some people in law enforcement are very helpful. e.g. human
trafficking rescuers generally have contacts in law enforcement. i
hear (and have experienced) that it can also be very dangerous to work
with law enforcement.
here is quote from notes thought slightly on:
re
- gptj uses a pregenerated constant causal mask that is O(n^2). since
it is simply a constant function of sequence index it could be made
via a callback or inside a loop.
- in perceiver, the user-provided attention vector is expanded with
1-length dimensions and passed on.
so perceiver has an O(n) attention mask. i didn't note a
model-associated bias. my code generates a bias to accommodate feature
matching between the two codebases, which will need an improvement
now let's review perceiver and/or gpt-j and see if the masks and
biases are O(n)able
it's 2022-02-02 10:31 UTC .
i wrote something like this in a state of mind:
a guess is that existing workers also value grassrootsness in
their hearts,
and could be organising the research such
https://github.com/AminRezaei0x443/memory-efficient-attention/pull/4
commit 1e45f724d55c938f991a483fc4ca9a4ac413b981
so, torch tensors are views, but jax tensors are copies.
- my current work was torch only so it is << O(n^2) if and only if the
passed matrices are not full and dense
- the jax code in memorty-efficient-attention has a bug, it can't be
<
Article summary:
Mobile phones apparently provide an API to track when a screenshot is
taken of an app. Facebook Messenger is planning to report to the other
user when a screenshot is taken.
It's helpful to have an airplane-mode burner phone to take emergency
pics with, reduces complications.
jim bell uses smartnews.com for their news. i'm not familiar with it,
looks convenient.
the pasted link is a tracking link for their algorithms. here's with it removed:
https://www.mirror.co.uk/news/weird-news/mark-zuckerberg-warns-against-taking-26106703
note mirror.co.uk will still track
On 2/2/22, grarpamp wrote:
> https://www.youtube.com/watch?v=F7cecQsuHKA Another censored conversation
Youtube uses AI to keep you clicking. Wake up and put your cocaine down.
> Globalist power is coming for your freedom and
> that of people you care about and around you.
> All of the signs
Good morning, spamthread.
I commented on the PR. I believe the DRY concern relates to
researchers being able to quickly review implementations without
having to switch files, unsure.
Here's the PR log:
2 days ago, xloem:
# What does this PR do?
This begins the implementation of a central
so far the ways to make expanded or repeated jax tensors have all made
copies for me. a test froze up this system for hours ;p
be well human race
- torch tensors do not allocate new memory when expanded, and are
documented as views that have that property
next: jax tensors, and model logic
arright! this is hard!
maybe we can expand some test tensor to be really big and see if
memory allocation changes
oops! it may be that when masks and biases are expanded to be dense no
additional memory is actually allocated.
uhhh !
re the masks and biases, basically the chunking code assumes they are
dense matrices, but by changing the chunking code you can pass only
the data needed. i'm presently doing that. it may end up that the
optimization is not reasonable on models that store a dense mask or
bias as an on-disk weight.
i'm working on below extant issue atm
also huggingface replied to the PR i made when i was losin' it, and
mentioned two other efficient attention implementations; they looked
approximation-based. also they said their repo is specifically
anti-DRY. which is not something anybody expects to hear.
uhhh the discord i remember the best is eleutherai's. they made gptj
and also an open source coding assistant app for vscode.
Note: I won't be effective at using the cutting edge here, because I
am not hanging in research chats on discord collaborating with
researchers sharing their latest work. Anybody can do that by hopping
through the chat servers, asking around. It feels a little
overwhelming for me.
Another idea:
We could design something using human knowledge or ghidra, then review it
and figure out how it could have designed it on its own.
Spamlog for understanding optimizers.
To move toward training models to optimize and design other models.
I'm thinking I'd like to try training a bytestokenizer for bigbird and
extend its sequence length to entire binaries. I expect the result to be
about 30% successful given my lack of experience and time.
idea: a model could be trained to guess the source layout by sequentially
producing filepaths and selecting areas of the source code to consider,
like an agent
that's similar to language generation except the output words/phrases are
unordered: a set of filepaths.
might be interesting to try
- I skimmed bigbird's description a little. it's trained for sequence
lengths of 4096 tokens but it doesn't look like memory requirements would
rise too much if that were increased somehow. curious if you can finetune a
model with increased position embeddings, probably can.
- I glanced at realm
- a large pretrained model that has significant understanding of
english logic and knowledge could be finetuned on bytes by training
perceiver-like cross attention embedding/tokenization encoders and
decoders to match the behaviors if its original tokenizer and
embeddings but accept byte streams.
Amin Rezaei commented on my work on their github, and pointed out that the
paper advises the technique is only useful on models with incredibly large
input data sizes. Not any of the ones I added it to.
Briefly thinking about that, it could be because of the size of the O(n^2)
data. For example,
Gunnar, did you resolve this?
I don't know Stacks, but if it's the kind of blockchain I'm used to it
would have a public ledger holding immutable NYCCoin balance history that
can be reviewed.
We can probably find time to help find that if it's hard for you.
The email was regarding a caving trip. On queue: verify that aligns with
an email I sent them. It likely does.
It is regrettable this thread is all caps and unlabeled with [spam] or [ot].
I received a reply to an email sent with this name, to the name "k". Raises
more interestingness.
I might put this goal on hold to recover. It is expressed in an aggressive
way, showing it's not fully developed, but
both tests passing, now to normalise things a bit. might be a little
hard to rewrite the torch scan() shim to match the approach that works
with jax
commit b60bc067e16c717fc6632d862f1de275007aa47e (HEAD ->
return_weights, origin/return_weights)
Date: Fri Jan 28 05:43:46 2022 +
jax
ok, the jax implementation will need a little rejiggering because jax
arrays are immutable, so passing one as an output parameter does not
provide for output.
grarpamp's description of a global code repository was really
inspiring. i'm wondering where and how to do work on it, like some
boss comes down with powersuit to playground.
hydraulic steps leave giant tracks in gravel as boss walks toward a
kid who looks a little like an employee boss didn't like.
boss raises hydraulic arm to punch poor child, about to kill them
all the parents are there
vivisectee powering
edited issue text, i think the flag is called 'output_attentions' or
something, not 'return_weights':
feat: output_attentions #1
I'm looking into hacking some of the models in the transformers
library to use this library for attention, and I don't see a way to
support `output_attentions` yet.
commit 899f4a6781537568b9b1b51250e7410c06716e9c
Author: xloem <0xl...@gmail.com>
Date: Thu Jan 27 16:47:08 2022 +
change dynamic_slice to reference passed data. return_weights test
now passes on torch.
commit 8de9d4c305e1763b2d0e90928b68d155ed60426c (HEAD ->
return_weights,
The toaster's estimated delivery date is Feb 11. I'll likely have
forgotten about it by that date, and have developed new inhibition
around engaging it.
To repeat, the age signature scheme I posted is totally bogus and
provides no cryptographic guarantees at all. A better one could likely
be
https://github.com/AminRezaei0x443/memory-efficient-attention/issues/1
feat: return_weights #1
xloem opened this issue 36 minutes ago
I'm looking into hacking some of the models in the transformers
library to use this library for attention, and I don't see a way to
support return_weights yet.
i've forked memory-efficient-attention in an attempt to add a
return_weights parameter. i think the torch implementation of this
would be simplified by using a for loop rather than a scan function
parameterised by a callback.
this commit finally produces the correct perceiver output without bailing.
i might like to do the gpt2 model next. since i ran into a lot of
unexpected difficulties here.
commit ab72f4a6a2a9095587b02c262ae1b20801172315 (HEAD ->
memory-efficient-attention, xloem/memory-efficient-attention)
Author:
the big issue wasn't truncation; it was that i had put the wrong block
of code in an if/else condition. current challenge is that the
efficient attention implementation doesn't provide for applying
dropout (random zeroing of some weights during training) where the
perceiver model applies it. i
i'm guessing the truncation happens in the chunking code. maybe there
is a permutation of the tensors that produces the right results but
with the wrong variable names, that i've stumbled upon.
the data comes out right now until it's consolidated at the end of the softmax
i stepped through it carefully, and it turns out the attention values
are being generated in a truncated manner. there are only 20 in the
efficient_attention code, whereas there are 96 in the working code.
so, i
bought it.
note: I also explored using a dremel to cut through the shielding
here. that looks like it would work but also takes fine placement.
the biggest issue is that it spews tiny shards of metal everywhere in
the process of cutting, and is very loud. i'm a guest in this indoors
environment,
ouch the cheapest toaster oven on amazon is like $90. i expected there
to be some cheap variant for like 20$ that broke after 4 uses.
i have a heat gun but it doesn't get hot enough to desolder the
high-temp solder used on the shielding on this phone. similarly, my
soldering iron has a similar
I'd like to put my age encryption key on an airgapped device.
Soemthing I've pursued a lot.
I'd like to remove the radios from an unopened phone to do this.
I have a phone to practice with. I'm not sure where I'm at with it: I
think I started, trying hard, and then stopped, maybe due to how hard
If I have angered you, I beg you to tell me clearly what I do not
understand here. I intend no harm. I do not understand this situation.
[spam][crazy] wow! that email containing the 'signatures' was hard to send.
also, my age already had the -y option. it's on age-keygen, not age.
I'm sure grarpamp's age signature scheme is actually for real quite
workable if one went into the age encryption aglorithms and properly
shared key
AGE-SECRET-KEY-1WURJSN9F7EH4AZCSS9ZGJKC3Y0KJLZ0CH734LTVEQUFPECLRYWDSCEZL8S
You can sign messages to this key via:
cat > message
sha256sum message | age --armor -r $(echo
AGE-SECRET-KEY-1WURJSN9F7EH4AZCSS9ZGJKC3Y0KJLZ0CH734LTVEQUFPECLRYWDSCEZL8S
| age-keygen -y) > signature
This encrypts the
Epilog:
The correct commit is
https://github.com/FiloSottile/age/commit/f3fdc33c9efcbe5c83e99751eac7097df95c4973
. The developer rewrote it, ref
https://github.com/FiloSottile/age/pull/146#issuecomment-795226571 .
grarpamp's email address proposed a way to sign messages using age.
You publicise a private key, and encrypt hashes to it.
age ostensibly has a way to generate public keys from private keys, in
this commit, haven't looked:
I received this private reply from you that doesn't look right and
ignores the encrypted message in the same email.
On 1/27/22, grarpamp wrote:
> On 1/23/22, k wrote:
>> Receiving an encrypted message doesn't indicate
>> the sender is the same
>> person who encrypted previous messages at all
>
what kind of chain do you imagine backing it? would the data be ipfs,
on-chain, something else?
the most important attribute of this project is working code, really,
imo. it's not a complex task, i just don't know where it's actually
been implemented.
Hey,
It's hard for me to read this in its entirety, but I wanted to relate
that most of the blockchain work on humaneness is in altcoins, and
that there are a number that engage various protocols of fairness. I
see the biggest issue as public information access: people don't learn
about these
aaand ... the notes regard commit
a021666abab736b7d98cd3d74712601bcf3aedf4 of
https://github.com/xloem/transformers in the
memory-efficient-attention branch, commit message 'wip'
these are my incomplete model permutation notes, for inside the
attention implementations. each axis is labeled with an einsum
letter.
chunked:
queries: ...qhd
keys: ...khd
values: ...vhd
mask: ...hqk -> ...qhk
i'm working on the matrix permutations inside huggingface perceiver
and the major torch implementation of efficient attention. i have two
scripts to call them, to step through and map the offsets. it's very
hard for me to think about the axis permutations.
scripts are attached.
>> "...even the NSA was quoted well over 10+ years ago saying that
>> the NSA could exploit tor."
>> Could you provide a link so I can share?
>
> search: "Tor Stinks" NSA presentation / slide deck
>
> https://www.theguardian.com/world/interactive/2013/oct/04/tor-stinks-nsa-presentation-document
The flickering entity returns.
Then the chair does too.
Chair: "We need you to help stop this!"
Doorknob-raining-spleens: "Sorry! I was hired to enter this data and I
need to finish by 3 o'clock!"
The intern's eyes widen.
Boss's Office During a Disagreement
Boss is nowhere to be seen. His body has turned into clouds of matter
flying everywhere. Chairs are upturned. Water, "rain" is flying
skyward amidst high plasma winds. An intern hides under somebody's
desk.
Intern: "What is going on"
They ask the
> https://bafkreie7gyyy3alribjyl72hlm4pk4allyul7xem7yqmpl66yzcidumfnq.ipfs.dweb.link/
I modified this to save the trained model with a line at the end of
"model.save_pretrained('words2nums')" which makes a folder and pickles
the trained parameters into it, in a torch-specific zip format.
I also
-
https://github.com/xloem/transformers/commit/7575b8286dd5c2b328d3c34d9b66dab434282fc0
A draft of calling memory_efficient_attention from the perceiver
model, when configuration parameters are set.
-
Untested. Maybe I can copy google's example again, like before,
somehow, and run the same test
i signed an email something like this:
apologies about the email name regarding abuse. it's part of a deal
with myself to direct my behavior better after experiencing coercion
to stay silent about political targeting.
email kind of changed :)
very hard. low chance.
79 matches
Mail list logo
