There are a couple.
1. visually displaying binary data more consisely than base64.
That problem is solved in the solution I posted. 256 printable elements have
been found. That was the only requirement. A clearly better solution would be
to simply copy ascii values of 32-127 along with box drawing characters into
that set. You might copy everything email-printable from this list into its
table location.
u: i.8 32 NB. will be many missing characters below (more than are missing in
jqt)
┌┬┐├┼┤└┴┘│─
!"#$%&'()*+,-./0123456789:;<=>?
@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_
`abcdefghijklmnopqrstuvwxyz{|}~
¡¢£¤¥¦§¨©ª«¬®¯°±²³´µ¶·¸¹º»¼½¾¿
ÀÁÂÃÄÅÆÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖ×ØÙÚÛÜÝÞß
àáâãäåæçèéêëìíîïðñòóôõö÷øùúûüýþÿ
In Doing so, you have a binary (base256) format that prints exactly the same as
ascii, if only ascii is used. You can edit your binary data with a text editor
and linefeeds/tabs are kept. There is little to no change required by
programming tools to allow editing binary data.
2. Replacement of ASCII text format, as a binary byte format with just LF CR
TAB maybe ESC as control flow elements.
You can think as codepages/character sets being 155 or so characters long, with
the common ascii characters (and box drawing) replicated in every page.
Codepages being used to communicate in any language while still fundamentally
using an ascii compatible language. Can also be used to display binary data
with symbols you recognize.
3. combining the above 2 for programming langage such as J. Extending the
potential primitives.
Room for 155 more base primitives. I understand that the core language is
based on the raw ascii code. It would extend possibilities. Editing files is
still a binary/text hybrid one byte per character. The only thing stored is
the char value, and users can use any codepage they feel like in order to
visually represent the symbols.
Ways to fill those 155 new primitives:
have separate monadic/dyadic only symbols for each primitive.
@:[ and @:] though simple enough, are line noisy, and special code can be
applied to the applied verb that predetermines it to be monadic. Also colour
formatting.
The changes would be forward (and easily backward) compatible with previous J
versions. As long as there is a J version for any new primitive, its machine
translatable to older J versions. Old J files would be 100% compatible.
Easier distinction between monads and dyads would be a great readability aid,
and probably what makes J hard for beginners to get into.
One language change worth considering would be to remove this domain error,
4 (3 : 'i. y') 3
replacing it with this behaviour:
4 i.@] 3
0 1 2
----- Original Message -----
From: Don Guinn <[email protected]>
To: Programming forum <[email protected]>
Cc:
Sent: Tuesday, July 7, 2015 1:28 PM
Subject: Re: [Jprogramming] interesting unicode
What is the problem you're trying to fix?
On Tue, Jul 7, 2015 at 11:17 AM, 'Pascal Jasmin' via Programming <
[email protected]> wrote:
> An interesting concept is base256 encoding that is visually useful.
>
> As a simple test
>
> u: 256 + i.8 32
>
> ĀāĂ㥹ĆćĈĉĊċČčĎďĐđĒēĔĕĖėĘęĚěĜĝĞğ
> ĠġĢģĤĥĦħĨĩĪīĬĭĮįİıIJijĴĵĶķĸĹĺĻļĽľĿ
> ŀŁłŃńŅņŇňʼnŊŋŌōŎŏŐőŒœŔŕŖŗŘřŚśŜŝŞş
> ŠšŢţŤťŦŧŨũŪūŬŭŮůŰűŲųŴŵŶŷŸŹźŻżŽžſ
> ƀƁƂƃƄƅƆƇƈƉƊƋƌƍƎƏƐƑƒƓƔƕƖƗƘƙƚƛƜƝƞƟ
> ƠơƢƣƤƥƦƧƨƩƪƫƬƭƮƯưƱƲƳƴƵƶƷƸƹƺƻƼƽƾƿ
> ǀǁǂǃDŽDždžLJLjljNJNjnjǍǎǏǐǑǒǓǔǕǖǗǘǙǚǛǜǝǞǟ
> ǠǡǢǣǤǥǦǧǨǩǪǫǬǭǮǯǰDZDzdzǴǵǶǷǸǹǺǻǼǽǾǿ
>
>
> will hopefully show up in all modern email clients. The problem with
> using the "superascii" code page is that it contains unprintable characters.
>
> while the above is the most computer efficient implementation of base256
> display, excellent other choices would be:
>
> 1. replace only the unprintable superascii characters
> 2. do so in a way that doesn't mess up J's boxing display.
> 3. still show up in any email client/browser from this decade.
> 4. Substitute with an eye for flexibility of unprintables such that
> greek/cyrillic/arabic/apl/math can be rendered (alongside "core
> superascii") one at a time. There may (I have no idea though japanese
> seems to use 81 phoneme characters) be some useful characters that can be
> further interpreted by asian languages as doubles or triples.
>
>
> while adding and subtracting 256 is dead simple conversion for encoding,
> it is not rocket science to handcraft a substitution table.
>
> The main purpose of this is to just display binary data. the convenience
> of encoding text into bytes with just a "document code page" overhead.
>
> Its also happens to be quite easy to take any list of integers and
> determine what code page it belongs to (as long as there is common
> superascii) if the integers include greek or japanese characters, then
> that is the writer's code page. The reader can display in any code page if
> all of the foreign characters "are greek to them"
>
> Has this idea already been implemented somewhere?
>
>
>
> ----- Original Message -----
> From: 'Pascal Jasmin' via Programming <[email protected]>
> To: "[email protected]" <[email protected]>
> Cc:
> Sent: Tuesday, July 7, 2015 12:09 AM
> Subject: Re: [Jprogramming] interesting unicode
>
> some more functions
>
> uucp2 =: ucp`]`(ucp@:u:)@.(1 i.~ 2 131072 = 3!:0)"1 :.utf82
> utf82 =: [: ": uucp2`(uucp2 ::u:@:({&a.))@.([: *./ 256&>)^:(1 4 64 e.~
> 3!:0)"1 :.uucp2
> futf =: 3 u: ":@:uucp2 :.fucp
> fucp =: 3 u: uucp2 :.futf
>
> the variants of uucp and utf8 take as input strings, unicode or integers,
> and ensure widechars or utf8 format.
>
> futf takes 3 types of input and returns utf8 bytes.
> fucp takes 3 types of input and returns unicode shorts.
>
> The only flakey part is converting from utf8 bytes directly to unicode
> shorts. If you don't know whether input integers are utf8 or utf16, you
> can use utf82 first as in: uucp2@:utf82
>
> > uucp2@:utf82 each futf each fucp each utf82 each futf each uucp2
> each <@utf82"1 i.8 32
>
> > uucp2@:utf82 each fucp each fucp each futf each <@utf82"1 ] 0 10000
> 20000 +/ i.8 32
>
> NB. ;: doesn't work well with utf. though cut is ok
>
> ;: '123 34 44 asdf' , , ' ' , "1 uucp2 2 }. i.8 24
>
> fucp each cut utf82 '123 34 44 asdf' , , ' ' , "1 uucp2 4 }. i.8 24
>
> ----- Original Message -----
> From: 'Pascal Jasmin' via Programming <[email protected]>
> To: Programming Forum <[email protected]>
> Cc:
> Sent: Monday, July 6, 2015 7:46 PM
> Subject: [Jprogramming] interesting unicode
>
> u: 256#. inv
> 9812938749123747923849273491287349128437921837928173492734791823749102983479128379128472342342342342342342342343x
>
> will print visually useful binary data (I guess referred to as superascii
> here:
>
> http://www.jsoftware.com/jwiki/Guides/UnicodeGettingStarted?highlight=%28unicode%29
> )
>
> timespacex 'u: 65536#. inv
> 9812938749123747923849273491287349128437921837928173492734791823749102983479128379128472342342342342342342342343x'
> 0.00072128 77184
>
> timespacex 'u: 256#. inv
> 9812938749123747923849273491287349128437921837928173492734791823749102983479128379128472342342342342342342342343x'
> 0.00013824 114944
>
> quite a bit faster to just stick to superascii, but the reason for it is
> this surprising result:
>
> timespacex '256x#. 256#. inv
> 9812938749123747923849273491287349128437921837928173492734791823749102983479128379128472342342342342342342342343x'
> 0.00016256 115200
>
>
> timespacex '65536x#. 65536#. inv
> 9812938749123747923849273491287349128437921837928173492734791823749102983479128379128472342342342342342342342343x'
> 0.00069696 77440
>
> its surprising in that it should be less work to split and reassemble into
> larger chunks.
>
> another quirk:
> (expected)
> 3!:0 u: 65536 #. inv 9812923438x
> 131072
>
> 3!:0 ": u: 65536 #. inv 9812923438x
> 2 NB. string but displays the same as if unicode. sounds neat.
>
> 3 u: u: ": u: 65536 #. inv 9812923438x
> 2 228 163 165 229 144 174
>
> if you take a string that looks like unicode it turns into superascii
>
> 3 u: u: 65536 #. inv 9812923438x
> 2 18661 21550
>
>
> one way to ensure the right result is to use instead of uucp,
>
>
> uucp2 =: ucp`]`(ucp@:u:)@.(1 i.~ 2 131072 = 3!:0)
>
>
> 3 u: uucp2 3 u: u: 65536 #. inv 9812923438x
> 2 18661 21550
> 3 u: uucp2 ": u: 65536 #. inv 9812923438x
> 2 18661 21550
>
> 3 u: uucp2 u: 65536 #. inv 9812923438x
> 2 18661 21550
> ----------------------------------------------------------------------
> For information about J forums see http://www.jsoftware.com/forums.htm
>
> ----------------------------------------------------------------------
> For information about J forums see http://www.jsoftware.com/forums.htm
> ----------------------------------------------------------------------
> For information about J forums see http://www.jsoftware.com/forums.htm
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
