Hi Folks, this is tangentially related because it is the reason for developing the x86 assembler and disassembler library.
introducing a new RISC instruction set with a variable length instruction packet using a super regular extension scheme with compression techniques to separate the instruction stream into two streams, one for instructions and and one for constants: - latest: https://metaparadigm.com/~mclark/glyph.pdf - current: https://metaparadigm.com/~mclark/glyph-20250512.pdf - emulator: https://github.com/michaeljclark/glyph Michael. On 5/14/25 19:39, Michael Clark wrote:
a port of an embeddable x86-mini disassembler to QEMU. - https://github.com/michaeljclark/qemu/tree/x86-mini-v3 # x86-mini disassembler the x86-mini library is a lightweight x86 encoder, decoder, and disassembler that uses extensions to the Intel instruction set metadata format to encode modern VEX/EVEX instructions and legacy instructions using a parameterized LEX (legacy extension) format. - metadata-driven disassembler with Intel format output. - written in C11 for compatibility with projects written in C. - low-level instruction encoder and decoder uses <= 32-bytes. - python tablegen program to generate C tables from CSV metadata. - metadata table tool to inspect operand encode and decode tables. - carefully checked machine-readable instruction set metadata. - support for REX/VEX/EVEX and preliminary support for REX2. the x86-mini x86 encoder and decoder library has been written from scratch to be modern and as simple as possible while also covering recent additions to the Intel and AMD 64-bit instruction sets such as the EVEX encodings for recent AVX-512 extensions and soon REX2/ EVEX encodings for Intel APX, as it is written with that in mind. ## interest to the QEMU community - x86-mini is fast. raw decode performance is ~100-200MiB/sec. - x86-mini is small. 5 files, ~5 KLOC or ~13 KLOC including tables. - x86-mini is complete and includes the latest AVX-512 extensions. - x86-mini is easy to extend and uses extended Intel format metadata. - x86-mini is documented with detailed info on the metadata format. - x86-mini has CLI tools for searching x86 instruction set metadata. it would be nice to have an x86 disassembler building out-of-the-box as I find QEMU's built-in tracing extremely useful and given x86 is a popular target, a small embedded disassembler might be useful. - https://github.com/michaeljclark/x86/tree/x86-mini ## techinical notes - the decoder is table-based and uses a metadata interpreter. - the decode table is ~66KiB with a ~150KiB acceleration trie. - there are currently 3658 opcode entries active on x86-64 which expands to 4775 table entries due to parameterization. after cherry-picking the commits, one can test host and target disassembly support. e.g. for an x86-64 target on an x86-64 host: $ echo aaa | qemu-x86_64 -d in_asm,out_asm /usr/bin/openssl sha256 ## caveats - supports 32-bit and 64-bit disassembly, and theoretically 16-bit. - designed to support 16-bit. base index formats are not done yet. - x86-64 is exhaustively fuzz-tested against the LLVM disassembler. - but x86-mini is new and hasn't been battle-tested in production. ## conclusion the metadata is based on x86-csv but has had numerous inaccuracies fixed plus conversion of legacy instructions to the new LEX format. in effect the metadata has been fuzz-tested against LLVM for x86-64 and ISA coverage is in the order of ~99.9%. the main branch of the linked repo has a procedural fuzzer for metadata-based instruction synthesis that could be useful for generating test cases for QEMU. ## summary of changes v3 - add SPDX license identifier to source files. - fix some typographical mistakes in header comments. - add control register, debug register, and MSR enumerations. - add x86_flag enumeration values for rflags register. - removed custom snprintf. v2 - improve grouping of encode, decode, and disasm functions. - fix some metadata documentation errors and ommissions. - use g_malloc/g_malloc0/g_free instead of malloc/calloc/free. - use cpu_to_{le16|le32|le64}, {le16|le32|le64}_to_cpu. - move bitfield documentation comments next to definitions. - redefine condition codes enum and remove redundant functions. - reduce snprintf performance cost with optimized version. - remove duplicate unused definition of ctz/clz/popcnt. - remove duplicate host endianness conversion functions. - remove some magic constants and replace with math. - remove some unnecessary casts. v1 - simplification to memory operand formatting logic. - fixed VSIB pointer formatting for consistency with LLVM. - 64-bit disasembly exhaustively fuzz tested against LLVM. - includes x86 metadata and x86-tablegen.py python script. - includes x86 metadata opcode and operand documentation. - addressed non-whitespace related checkpatch.pl warnings. Michael Clark (4): x86-disas: add x86-mini instruction set metadata x86-disas: add x86-mini metadata documentation x86-disas: add x86-mini metadata tablegen script x86-disas: add x86-mini disassembler implementation disas/disas-host.c | 5 + disas/meson.build | 97 + disas/x86-core.c | 2716 +++++++++++++++++++++ disas/x86-data/x86_adx.csv | 5 + disas/x86-data/x86_aes.csv | 13 + disas/x86-data/x86_aeskle.csv | 3 + disas/x86-data/x86_aesklewide.csv | 3 + disas/x86-data/x86_avx.csv | 375 +++ disas/x86-data/x86_avx2.csv | 171 ++ disas/x86-data/x86_avx5124fmaps.csv | 5 + disas/x86-data/x86_avx5124vnniw.csv | 3 + disas/x86-data/x86_avx512b.csv | 3 + disas/x86-data/x86_avx512bf16.csv | 10 + disas/x86-data/x86_avx512bitalg.csv | 10 + disas/x86-data/x86_avx512bw.csv | 291 +++ disas/x86-data/x86_avx512cd.csv | 19 + disas/x86-data/x86_avx512d.csv | 2 + disas/x86-data/x86_avx512dq.csv | 143 ++ disas/x86-data/x86_avx512er.csv | 11 + disas/x86-data/x86_avx512f.csv | 901 +++++++ disas/x86-data/x86_avx512fp16.csv | 237 ++ disas/x86-data/x86_avx512ifma.csv | 7 + disas/x86-data/x86_avx512pf.csv | 17 + disas/x86-data/x86_avx512vbmi.csv | 13 + disas/x86-data/x86_avx512vbmi2.csv | 61 + disas/x86-data/x86_avx512vl.csv | 2 + disas/x86-data/x86_avx512vnni.csv | 13 + disas/x86-data/x86_avx512vp2intersect.csv | 7 + disas/x86-data/x86_avx512vpopcntdq.csv | 7 + disas/x86-data/x86_avxneconvert.csv | 15 + disas/x86-data/x86_avxvnni.csv | 9 + disas/x86-data/x86_avxvnniint8.csv | 13 + disas/x86-data/x86_base.csv | 549 +++++ disas/x86-data/x86_bmi1.csv | 13 + disas/x86-data/x86_bmi2.csv | 17 + disas/x86-data/x86_cet.csv | 15 + disas/x86-data/x86_cldemote.csv | 2 + disas/x86-data/x86_clwb.csv | 2 + disas/x86-data/x86_enqcmd.csv | 3 + disas/x86-data/x86_f16c.csv | 5 + disas/x86-data/x86_fma.csv | 97 + disas/x86-data/x86_fsgsbase.csv | 9 + disas/x86-data/x86_gfni.csv | 19 + disas/x86-data/x86_hreset.csv | 2 + disas/x86-data/x86_invpcid.csv | 2 + disas/x86-data/x86_lzcnt.csv | 3 + disas/x86-data/x86_mmx.csv | 60 + disas/x86-data/x86_movdir64b.csv | 2 + disas/x86-data/x86_movdiri.csv | 3 + disas/x86-data/x86_mpx.csv | 9 + disas/x86-data/x86_msrlist.csv | 4 + disas/x86-data/x86_ospke.csv | 3 + disas/x86-data/x86_pclmulqdq.csv | 3 + disas/x86-data/x86_pconfig.csv | 2 + disas/x86-data/x86_prefetchw.csv | 7 + disas/x86-data/x86_raoint.csv | 9 + disas/x86-data/x86_rdpid.csv | 2 + disas/x86-data/x86_rdrand.csv | 2 + disas/x86-data/x86_rdseed.csv | 2 + disas/x86-data/x86_rtm.csv | 5 + disas/x86-data/x86_serialize.csv | 2 + disas/x86-data/x86_sha.csv | 8 + disas/x86-data/x86_smap.csv | 3 + disas/x86-data/x86_sse.csv | 58 + disas/x86-data/x86_sse2.csv | 148 ++ disas/x86-data/x86_sse3.csv | 11 + disas/x86-data/x86_sse4_1.csv | 48 + disas/x86-data/x86_sse4_2.csv | 7 + disas/x86-data/x86_sse4_3.csv | 2 + disas/x86-data/x86_ssse3.csv | 33 + disas/x86-data/x86_uintr.csv | 6 + disas/x86-data/x86_vaes.csv | 17 + disas/x86-data/x86_vmx.csv | 14 + disas/x86-data/x86_waitpkg.csv | 4 + disas/x86-data/x86_wbnoinvd.csv | 2 + disas/x86-data/x86_x87.csv | 145 ++ disas/x86-data/x86_xsaveopt.csv | 3 + disas/x86-disas.c | 96 + disas/x86.h | 1860 ++++++++++++++ docs/x86-metadata.txt | 303 +++ include/disas/dis-asm.h | 1 + scripts/x86-tablegen.py | 695 ++++++ target/i386/cpu.c | 7 + 83 files changed, 9511 insertions(+) create mode 100644 disas/x86-core.c create mode 100644 disas/x86-data/x86_adx.csv create mode 100644 disas/x86-data/x86_aes.csv create mode 100644 disas/x86-data/x86_aeskle.csv create mode 100644 disas/x86-data/x86_aesklewide.csv create mode 100644 disas/x86-data/x86_avx.csv create mode 100644 disas/x86-data/x86_avx2.csv create mode 100644 disas/x86-data/x86_avx5124fmaps.csv create mode 100644 disas/x86-data/x86_avx5124vnniw.csv create mode 100644 disas/x86-data/x86_avx512b.csv create mode 100644 disas/x86-data/x86_avx512bf16.csv create mode 100644 disas/x86-data/x86_avx512bitalg.csv create mode 100644 disas/x86-data/x86_avx512bw.csv create mode 100644 disas/x86-data/x86_avx512cd.csv create mode 100644 disas/x86-data/x86_avx512d.csv create mode 100644 disas/x86-data/x86_avx512dq.csv create mode 100644 disas/x86-data/x86_avx512er.csv create mode 100644 disas/x86-data/x86_avx512f.csv create mode 100644 disas/x86-data/x86_avx512fp16.csv create mode 100644 disas/x86-data/x86_avx512ifma.csv create mode 100644 disas/x86-data/x86_avx512pf.csv create mode 100644 disas/x86-data/x86_avx512vbmi.csv create mode 100644 disas/x86-data/x86_avx512vbmi2.csv create mode 100644 disas/x86-data/x86_avx512vl.csv create mode 100644 disas/x86-data/x86_avx512vnni.csv create mode 100644 disas/x86-data/x86_avx512vp2intersect.csv create mode 100644 disas/x86-data/x86_avx512vpopcntdq.csv create mode 100644 disas/x86-data/x86_avxneconvert.csv create mode 100644 disas/x86-data/x86_avxvnni.csv create mode 100644 disas/x86-data/x86_avxvnniint8.csv create mode 100644 disas/x86-data/x86_base.csv create mode 100644 disas/x86-data/x86_bmi1.csv create mode 100644 disas/x86-data/x86_bmi2.csv create mode 100644 disas/x86-data/x86_cet.csv create mode 100644 disas/x86-data/x86_cldemote.csv create mode 100644 disas/x86-data/x86_clwb.csv create mode 100644 disas/x86-data/x86_enqcmd.csv create mode 100644 disas/x86-data/x86_f16c.csv create mode 100644 disas/x86-data/x86_fma.csv create mode 100644 disas/x86-data/x86_fsgsbase.csv create mode 100644 disas/x86-data/x86_gfni.csv create mode 100644 disas/x86-data/x86_hreset.csv create mode 100644 disas/x86-data/x86_invpcid.csv create mode 100644 disas/x86-data/x86_lzcnt.csv create mode 100644 disas/x86-data/x86_mmx.csv create mode 100644 disas/x86-data/x86_movdir64b.csv create mode 100644 disas/x86-data/x86_movdiri.csv create mode 100644 disas/x86-data/x86_mpx.csv create mode 100644 disas/x86-data/x86_msrlist.csv create mode 100644 disas/x86-data/x86_ospke.csv create mode 100644 disas/x86-data/x86_pclmulqdq.csv create mode 100644 disas/x86-data/x86_pconfig.csv create mode 100644 disas/x86-data/x86_prefetchw.csv create mode 100644 disas/x86-data/x86_raoint.csv create mode 100644 disas/x86-data/x86_rdpid.csv create mode 100644 disas/x86-data/x86_rdrand.csv create mode 100644 disas/x86-data/x86_rdseed.csv create mode 100644 disas/x86-data/x86_rtm.csv create mode 100644 disas/x86-data/x86_serialize.csv create mode 100644 disas/x86-data/x86_sha.csv create mode 100644 disas/x86-data/x86_smap.csv create mode 100644 disas/x86-data/x86_sse.csv create mode 100644 disas/x86-data/x86_sse2.csv create mode 100644 disas/x86-data/x86_sse3.csv create mode 100644 disas/x86-data/x86_sse4_1.csv create mode 100644 disas/x86-data/x86_sse4_2.csv create mode 100644 disas/x86-data/x86_sse4_3.csv create mode 100644 disas/x86-data/x86_ssse3.csv create mode 100644 disas/x86-data/x86_uintr.csv create mode 100644 disas/x86-data/x86_vaes.csv create mode 100644 disas/x86-data/x86_vmx.csv create mode 100644 disas/x86-data/x86_waitpkg.csv create mode 100644 disas/x86-data/x86_wbnoinvd.csv create mode 100644 disas/x86-data/x86_x87.csv create mode 100644 disas/x86-data/x86_xsaveopt.csv create mode 100644 disas/x86-disas.c create mode 100644 disas/x86.h create mode 100644 docs/x86-metadata.txt create mode 100755 scripts/x86-tablegen.py
