barry-jin opened a new issue #20257: URL: https://github.com/apache/incubator-mxnet/issues/20257
## Problem statement Currently, migration of cpp-package to MXNet2.0 is [in progress](https://github.com/apache/incubator-mxnet/pull/20131). The changes in MXNet2.0 cpp-package include some tasks like renaming some CAPIs with Ex suffix, adopting CachedOp interface in executor forward computing and autograd interface in executor backward propogation. Which means, the v1.x cpp-package users can migrate to v2.0 to still utilize executor to do training or inferencing on their models. However, there are two limitations: 1. The cross-compilation problem is not resolved at all. In issue https://github.com/apache/incubator-mxnet/issues/13303 and issue https://github.com/apache/incubator-mxnet/issues/20222, users are still facing the issue from using [OpWrapperGenerator.py](https://github.com/apache/incubator-mxnet/blob/v1.8.x/cpp-package/scripts/OpWrapperGenerator.py) to open a target binary on the host machine. 2. The users for cpp-package are still interacting with symbols, which lacks certain flexibility in model construction. ## Proposed New C++ Frontend Based on the above limitations, I propose to adopt the design of gluon api to build a new version of cpp-package with both flexibility and performance benefits. The design logic is highly similar to current gluon2.0 with tracing and deferred compute. I have been building a simple gluon based [cpp-package](https://github.com/barry-jin/incubator-mxnet/tree/cpp2/cpp-package) and a [simple demo](https://github.com/barry-jin/incubator-mxnet/blob/cpp2/cpp-package/example/simple_demo.cpp) to create an end-to-end training process. The following is the high level overview of some APIs. ### [Operator](https://github.com/barry-jin/incubator-mxnet/blob/cpp2/cpp-package/include/mxnet-cpp/operator_rt.h) New operator is relying on packed function's runtime api(this is a runtime api, probably will solve the limitation1). Firstly, an [operator map](https://github.com/barry-jin/incubator-mxnet/blob/cpp2/cpp-package/include/mxnet-cpp/op_rt_map.h#L48-L69) will be created by mapping from the operator's string name to its function handle. Then user only need to use this [macro](https://github.com/barry-jin/incubator-mxnet/blob/07d3a73eaa7443ed1b211d6788515f275608350c/cpp-package/include/mxnet-cpp/operator_rt.h#L41-L52) to create the operator wrapper in the `mxnet::cpp::op` namespace. Inside the operator, the [type conversion and translation](https://github.com/barry-jin/incubator-mxnet/blob/07d3a73eaa7443ed1b211d6788515f275608350c/cpp-package/include/mxnet-cpp/operator_rt.h#L89-L165) happens. It's similar to the [function.py](https://github.com/apache/incubator-mxnet/blob/970a2cfbe77d09ee610fdd70afca1a93247cf4fb/python/mxnet/_ffi/_ctypes/function.py#L56-L96), which erase the type informa tion for each argument and wrapped into MXNetValue, passed to C++ backend. ### [Block](https://github.com/barry-jin/incubator-mxnet/blob/cpp2/cpp-package/include/mxnet-cpp/block.h) MXNet C++ block will be the base class of all the basic layers, which is the same as python. It will use [register_block](https://github.com/barry-jin/incubator-mxnet/blob/07d3a73eaa7443ed1b211d6788515f275608350c/cpp-package/include/mxnet-cpp/block.h#L70-L71) to store the pointer to its children in a [vector](https://github.com/barry-jin/incubator-mxnet/blob/07d3a73eaa7443ed1b211d6788515f275608350c/cpp-package/include/mxnet-cpp/block.h#L350-L351). [Tracing and deferred compute](https://github.com/barry-jin/incubator-mxnet/blob/07d3a73eaa7443ed1b211d6788515f275608350c/cpp-package/include/mxnet-cpp/block.h#L312-L340) is used to create the graph. ### [basic_layers](https://github.com/barry-jin/incubator-mxnet/blob/cpp2/cpp-package/include/mxnet-cpp/nn/basic_layers.h) Currently, I have only Dense, Activation and Dropout layer for demo purpose. Parameters for dense layer will be registered with [register_parameter](https://github.com/barry-jin/incubator-mxnet/blob/07d3a73eaa7443ed1b211d6788515f275608350c/cpp-package/include/mxnet-cpp/nn/basic_layers.hpp#L53-L54) method. ### [autograd](https://github.com/barry-jin/incubator-mxnet/blob/cpp2/cpp-package/include/mxnet-cpp/autograd.h) Since there is a lack of context manager mechanism in C++, we have to use [`start_recording()`](https://github.com/barry-jin/incubator-mxnet/blob/cpp2/cpp-package/include/mxnet-cpp/autograd.h) and [`finish_recording()`](https://github.com/barry-jin/incubator-mxnet/blob/07d3a73eaa7443ed1b211d6788515f275608350c/cpp-package/include/mxnet-cpp/autograd.h#L76-L83) to define the autograd recording scope. Also, the [backward](https://github.com/barry-jin/incubator-mxnet/blob/07d3a73eaa7443ed1b211d6788515f275608350c/cpp-package/include/mxnet-cpp/autograd.h#L90-L96) method is a simple wrapper around the CAPI. ### Others There are some other simple modules, like [Trainer](https://github.com/barry-jin/incubator-mxnet/blob/cpp2/cpp-package/include/mxnet-cpp/trainer.h) and [loss](https://github.com/barry-jin/incubator-mxnet/blob/cpp2/cpp-package/include/mxnet-cpp/loss.h) ## Example ### A simple example ```C++ #include <chrono> #include <string> #include "utils.h" #include "mxnet-cpp/MxNetCpp.h" using namespace mxnet::cpp; // Define a new Model derived from Block class Model : public gluon::Block { public: Model() { // Register three dense blocks and one dropout block(need user to provide shape) dense0 = register_block("dense0", gluon::nn::Dense(64, 784, "relu")); dropout0 = register_block("dropout0", gluon::nn::Dropout(0.5)); dense1 = register_block("dense1", gluon::nn::Dense(32, 64, "sigmoid")); dense2 = register_block("dense2", gluon::nn::Dense(10, 32)); } // Model's forward algorithm NDArray forward(NDArray x) { x = dense0(x); x = dropout0(x); x = dense1(x); x = dense2(x); return x; } gluon::nn::Dense dense0, dense1, dense2; gluon::nn::Dropout dropout0; }; int main(int argc, char** argv) { // Create a new model Model model; // Use Uniform Initializer to initialize the model's parameters with scale of 0.07 model.initialize<Uniform>(Uniform(0.07)); // Use legacy cpp-package's dataloader int batch_size = 32; std::vector<std::string> data_files = { "./data/mnist_data/train-images-idx3-ubyte", "./data/mnist_data/train-labels-idx1-ubyte", "./data/mnist_data/t10k-images-idx3-ubyte", "./data/mnist_data/t10k-labels-idx1-ubyte" }; auto train_iter = MXDataIter("MNISTIter"); if (!setDataIter(&train_iter, "Train", data_files, batch_size)) { return 1; } // Define autograd AutoGrad ag(true, true); // Define trainer with sgd optimizer and 0.5 learning_rate std::unordered_map<std::string, double> opt_params; opt_params["lr"] = 0.5; Trainer trainer(model.collect_parameters(), "sgd", opt_params); // Define loss function gluon::SoftmaxCrossEntropyLoss loss_fn; for (size_t epoch = 1; epoch <= 100; ++epoch) { size_t batch_index = 0; // Reset train iter train_iter.Reset(); // Iterate the dataloader while (train_iter.Next()) { // Get data batch auto batch = train_iter.GetDataBatch(); // Start Autograd recording ag.start_recording(); // Apply model on the input data NDArray pred = model(batch.data); // Compute loss value NDArray loss = loss_fn(pred, batch.label); // Backward propagation on the loss ag.backward(loss); // Finish Autograd recording ag.finish_recording(); // Update parameters with trainer trainer.step(); NDArray::WaitAll(); if (++batch_index % 100 == 0) { std::cout << "Epoch: " << epoch << " | Batch: " << batch_index << " | Loss: " << loss.item<float>() << std::endl; } } } } ``` ## References [1] https://github.com/apache/incubator-mxnet/blob/master/python/mxnet/gluon/block.py -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
