Hi! I would like to share with you a hobby project I've been working on recently. The idea & motivation behind it is, to try and enable writing native (.apk) Android apps with Nim, without having to install and use Android Studio. What I want to share with you is the first major success on this path — a showcase JNI-based .apk, with Dalvik bytecode assembled purely with Nim, and the ARM code written in Nim and cross-compiled with Android NDK.
The project at this point of life is composed of the following elements: * [hellomello](https://github.com/akavel/hellomello) — a sample "hello world" project, with step-by-step instructions how to build it; * [dali](https://github.com/akavel/dali) — Dalvik Assembler/Linker — a small Nim package, implementing an assembler for Dalvik bytecode, emitting .dex files; (this repo currently also accidentally contains [part of the hellomello sources](https://github.com/akavel/dali/blob/adc2d2d6c1fb33aaac89d49e80a4f0ff4378fd10/jni_hello.nim#L76-L96), that should really be moved into the hellomello repo; sorry that I haven't cleaned this it up yet) * [marco](https://github.com/akavel/marco) — Manifest and Resource Compiler for .apk — a small Nim tool for compiling AndroidManifest.xml files from regular XML to a binary format required for .apk; * [apksigner](https://github.com/akavel/apksigner) — a small tool for cryptographically signing .apk files — _NOTE:_ this one is unfortunately currently written in Go; it was faster for me to prototype it this way, as I had trouble finding appropriate crypto library in Nim. I'd be **super grateful** if someone could help me with this part (I need the following primitives: [parsing a PKCS#8 file with priv. key](https://github.com/akavel/apksigner/blob/ae0fb98d2f84796f726f7fa3786bbe2115e7dd1a/apksigner.go#L56); [parsing an x509 PEM file with cert](https://github.com/akavel/apksigner/blob/ae0fb98d2f84796f726f7fa3786bbe2115e7dd1a/apksigner.go#L64-L71); signing a message [using a PKCS#7 signature/format](https://github.com/akavel/apksigner/blob/ae0fb98d2f84796f726f7fa3786bbe2115e7dd1a/apksigner.go#L166-L179)) — with your help, we could make this project 100% Nim-based. * [Android NDK](https://developer.android.com/ndk/downloads) — the cross-compilation toolchain (standalone variant, as is [the default](https://developer.android.com/ndk/guides/standalone_toolchain) now) — At this stage of the project, I managed to get rid of Android Studio/SDK, but we still need the NDK. Notably, you _can_ actually already live without Android NDK, as long as you plan to write your whole app purely in Dalvik assembly ([example — a hello-world](https://github.com/akavel/dali/blob/adc2d2d6c1fb33aaac89d49e80a4f0ff4378fd10/czak_hello.nim#L50-L60)). But if you prefer to write as much as possible in _Nim_ , you'll currently need to cross-compile it as a JNI library, and the NDK is what provides cross-compilers. The resulting .apk file works on my phone (a Samsung Galaxy A3 2017), displaying a simple "Hello world from Nim" message, as can be seen on the following (I'm afraid not very glamorous) screenshot: # Build Steps To reproduce the hellomello .apk, follow the steps listed in [the README of the hellomello project](https://github.com/akavel/hellomello). (Please tell me if something doesn't work for you. I tried to test/verify the instructions after listing them, but I could miss or break something.) # Various Notes **Completion status.** I must admit I find it surprisingly hard for me to adequately describe the status of the project at this moment. On one hand, it's kinda "alpha/prototype quality" — in that it's not a "complete, polished, high-level solution". Notably, if it _crashes_ on you (which may happen!), please don't be scared! I would basically treat it as a signal meaning: "(some part of) a functionality you need is not yet implemented; you're welcome to add it if you like!" On the other hand, I believe where it works, it just works. I mean, after you write some new app, you have to check if it works on an Android device, and fix any errors emitted by the Android verifier into the `adb logcat` console. But Dali is an assembler, which means it's low-level, and that's kinda how one works with assemblers. However, once you make your app install and open without Android verifier errors, I would expect it should just work thereafter. So, should I call it a "prototype"? an "MVP"? "Technology preview"? Not sure. **Minimal Android version.** So, that's a bit tricky. I basically don't know. I tested the resulting .apk on a device I own, which has Android 8. There are a few places where the specs required me to put some "minimal version", and I just used whatever value worked for me (from what I recall currently: [in the .dex header](https://github.com/akavel/dali/blob/adc2d2d6c1fb33aaac89d49e80a4f0ff4378fd10/src/dali.nim#L176) and [in manifest files](https://github.com/akavel/marco/blob/master/src/marcopkg/compile.nim#L32)). I have no idea if the .apk will work on earlier Androids, or not. Please have a look at the next paragraph as to why I myself don't plan to test with earlier versions. **Support.** It's important to note, that as of now, this is purely a hobby project for me. I'm very much **open to sponsoring /hire** to work on it full time (and actually, **now** would be the very best moment to contact me if you're interested, as I'm very actively looking for a new job — please email me); though unless that happens, I'm _very_ limited in time & resources I can allocate to this project. As such, **for the time being** , I'm afraid **I cannot afford to provide any support** for anyone interested in this project, **unless hired /sponsored** (see below). You're more than welcome to report any issues, questions, and send pull requests, and you can be sure I will read them, and heartfully appreciate them. But even for PRs, I cannot promise I will give them enough due attention; I may, or may not. I will try to at least respond with a "thank you" and a reminder of no promise of support. I'm sorry, I just cannot stretch my available time any more. The chances of me responding are probably highest for questions, as those are easiest to respond to. I'm especially happy to answer any questions in this thread on this forum over the next several days, though even here I might not be able to do that _quickly_ (may take me a few days). Please note you're **welcome to fork** this project, and modify it to your needs. My target goal is actually _writing some personal apps for Android_ , while _avoiding Java /JVM_, so if you make it possible with this project faster than I, please just let me know! **Future, plans, ideas.** To tell the truth, I'm currently not 100% decided where I want to take the project further. That's because there are really many possibilities, and I'm not sure which one to choose! :) One of the first ideas I had, would be to add .dex/.apk as a **backend for the Nim compiler**. Notably, I believe it could be surprisingly reasonable and simple, given how high-level the Dalvik bytecode is. Specifically, I believe it would be most similar to the _JS backend_ , even more than to the C backend, not even mentioning any kind of typical machine code. That's because in Dalvik bytecode, you don't have to worry about stuff like garbage collection, call stack management, or (mostly) register allocation (roughly, you can just allocate each local variable to one of the 65535 available function-local registers). I found the official [Dalvik bytecode reference](https://source.android.com/devices/tech/dalvik/dalvik-bytecode) very simple, readable and approachable. In case of doubt, [the smali wiki](https://github.com/JesusFreke/smali/wiki/Registers) clears up some remaining uncertainties. In collaboration with the awesome [Richard Jansson](https://github.com/richard-jansson) (I recommend you take a look at his [innovative text input methods project](http://sigma.eruditenow.com/index.html)!), we started some [initial experiment/exploration](https://github.com/dalinim/dalim/) in this direction. I believe it is a promising path, and should be relatively simple; but I also estimate it's _just_ too much for me to be able to pursue purely in my spare time. (If you're interested, as I already said, I'm open to hire/sponsoring — see below; alternatively, please feel free to fork this project.) A second avenue I'm pondering is **making dali more high-level** , a.k.a. a "macro assembler", potentially as a Nim DSL via macros. Maybe even taking it as far as some kind of half-baked language targeting Dalvik; maybe even Java-like? (Given that Java appears to be not really that far from the raw Dalvik bytecode.) A third direction I thought of would be to maybe actually **try and write some real apps in Nim with JNI** , and see how it works for my needs. Maybe try using nimx for GUI? Explore the status of touch events support for nimx and/or JNI apps in general? I would also love to get rid of the NDK in this case if possible; I started experimenting with maybe using Zig as a cross-compiler instead of the NDK, but I am [not there yet](https://github.com/ziglang/zig/issues/2437). Personally, I'm mainly interested in GUI apps. On the other hand, the current state of the project should be an especially good fit also for exploring writing **Android games in Nim**. I believe it might be not far from just changing the hellomello project to use a NativeActivity (in AndroidManifest.xml and in the .dex). But I don 't know anything about writing games for Android; so, either you'll have to explore this further on your own (but feel free to ask me on any aspects related to dali/marco/...), or, as I said, you can consider sponsoring me to help you — see below. **Sponsoring /hire.** If you're interested in hiring/sponsoring me to work on this project, and to take it further in some **direction interesting for both of us** , please **contact me** by email ([email protected]). I'd be very happy to get involved in such a collaboration. Please do it **ASAP** , as I'm currently in the process of searching for a new job. I'm super open and eager _now_ (we can agree on starting a bit later this year), but once I start working at some other place that I'll like, I won't expect to be open to switch any longer for quite some time. **References & internals — dali.** I wrote the [dali .dex assembler](https://github.com/akavel/dali) based primarily on an awesome [PDF with an annotated diagram of a hello-world .dex binary](https://github.com/corkami/pics/blob/3cb08bc92e6c7508a30ca0d9f5f8a3319a300591/binary/DalvikEXecutable.pdf), by Ange Albertini. I consulted the [official .dex spec](https://source.android.com/devices/tech/dalvik/dex-format.html) and the [official Dalvik bytecode spec](https://source.android.com/devices/tech/dalvik/dalvik-bytecode) along the way. The specs are _small_ , clear, readable and simple, so it was really fairly straightforward. I'm very happy that I initially said "let's try how far I can get," instead of worrying what could go wrong. Especially the Dalvik bytecode is surprisingly (to me) high level and reasonably easy to write. At high level, the main function rendering a .dex file is basically [155 lines long](https://github.com/akavel/dali/blob/adc2d2d6c1fb33aaac89d49e80a4f0ff4378fd10/src/dali.nim#L172-L327), while the _whole assembler, including types_ is only [600 lines of Nim, including comments](https://github.com/akavel/dali/blob/adc2d2d6c1fb33aaac89d49e80a4f0ff4378fd10/src/dali.nim). I'm very happy how Nim worked for me on this project. I see it as having a great balance of expressivity and readability, with static typing being of immense help, while also non-intrusive. **References & internals — .apk files.** I built my understanding of .apk files based on the [ApkGolf](https://github.com/fractalwrench/ApkGolf) project, and the [accompanying article](https://fractalwrench.co.uk/posts/playing-apk-golf-how-low-can-an-android-app-go/), which aims to construct a simplest possible .apk file. Thanks to this project and its author ([Jamie Lynch](https://github.com/fractalwrench/)), I managed to understand that there's not really much needed to package a .dex into an .apk. So I went on and wrote the rest of the necessary tools (marco & apksigner) in Nim. **References & internals — marco.** [This app](https://github.com/akavel/marco) currently can only compile the simplest AndroidManifest.xml files (notably, it can't emit binary resource files yet), but extending it should be easy enough. The project is currently [just 250 lines of Nim](https://github.com/akavel/marco/blob/f6213ec82ce43dae7092bcd573fc207f02b5231b/src/marcopkg/compile.nim), and I listed some most useful references I found [in the readme](https://github.com/akavel/marco#references). My own initial mode of operation was analyzing [a minimal binary .xml file](https://github.com/akavel/marco/blob/f6213ec82ce43dae7092bcd573fc207f02b5231b/tests/tdump.nim#L11), together with the sources of [a Java parser/clone of aapt](https://github.com/sdklite/aapt/blob/9e6d1ad98469dffbc9940821551bd7a2e07dd1e0/src/main/java/com/sdklite/aapt/AssetEditor.java) for reference. **References & internals — apksigner.** [This app](https://github.com/akavel/apksigner) is the simplest one at the high level, as it just takes a .zip file (containing classes.dex & binary AndroidManifest.xml), then calculates SHA1 hashes of the archived files, and signs them with a user-provided private key + cert. Finally, it writes the hashes & signature as a few new files in a `META-INF/` directory in the .zip. Unfortunately, to keep prototyping fast, I had to write it **in Go** , as I had trouble finding appropriate crypto libraries for Nim, and being a Nim newbie, I didn't know how to correctly wrap OpenSSL (?) for this either. In Go, I fortunately managed to quickly find some packages that worked for me, so I was able to quickly verify if writing this app makes sense, which was crucial at that time and phase. I'd be **super grateful** if someone could help me rewrite this **in Nim** now, at least by helping to get access to some **necessary crypto primitives**. Specifically (as listed earlier), I desperately need to be able to: * [parse a PKCS#8 file, to load a private key](https://github.com/akavel/apksigner/blob/ae0fb98d2f84796f726f7fa3786bbe2115e7dd1a/apksigner.go#L56) ; * [parse an x509 PEM file, to load a certificate](https://github.com/akavel/apksigner/blob/ae0fb98d2f84796f726f7fa3786bbe2115e7dd1a/apksigner.go#L64-L71); * sign a message [and emit a PKCS#7-formatted signature](https://github.com/akavel/apksigner/blob/ae0fb98d2f84796f726f7fa3786bbe2115e7dd1a/apksigner.go#L166-L179). **_Could you help me with that?_** **Nimble & hellomello.** Apart from some help porting apksigner to Nim, I would really appreciate if someone could help me make the sample [hellomello](https://github.com/akavel/hellomello/) project "self contained", with Nimble. I understand Nim uses NimScript for building, but I never used it, and am not sure where to even start and how to arrange it, between Nimble and nim.cfg. I would love if it could be possible to tell a new user to, say, "just clone hellomello and run `nimble build`", and have all the remaining steps done automatically by Nimble/NimScript. This way, I would hope forking hellomello could be made the whole procedure required to start a new Android project in Nim. **_Could you help me with that?_** (Notably, the [jni_hello.nim](https://github.com/akavel/dali/blob/adc2d2d6c1fb33aaac89d49e80a4f0ff4378fd10/jni_hello.nim#L76-L96) file, emitting the [bytecode wrapper for the JNI](https://github.com/akavel/dali/blob/adc2d2d6c1fb33aaac89d49e80a4f0ff4378fd10/jni_hello.nim#L76-L96) should be moved from the dali repository into the hellomello repo.) Other than that, I'm not sure how to best publish dali+marco+apksigner via Nimble. Should I publish them separately? Or would it maybe be a good idea to eventually merge them into a single tool, with subcommands? But dali is currently only a library... Or, maybe I shouldn't publish them yet to the Nimble directory, if they're still at a very early phase? **That 's all for now — hope you like it! :)**
