Deploy DeepSeek-R1 on RK3588 Development Board — Part 1

Overview

DeepSeek is a China-developed large AI model known for its inference capability and efficient text generation. The DeepSeek-R1 iteration improves long-text handling, plans for multimodal extensions, and adds embedded-device adaptation features.

The RK3588, a flagship SoC from Rockchip, offers heterogeneous multi-core compute with capable CPU, GPU, and NPU blocks, making it a suitable platform for embedded AI workloads. Deploying DeepSeek-R1 on the OK3588-C development board demonstrates how a large AI model can be moved from cloud-based deployment to edge devices, addressing latency and privacy requirements for on-device inference.

01 Porting Process

1. Download DeepSeek-R1 source and weights
   On an Ubuntu virtual machine, obtain the DeepSeek-R1-Distill-Qwen-1.5B weight files from the DeepSeek-R1 website.
2. Install conversion tools
   Create a Python virtual environment on Ubuntu and install the RKLLM-Toolkit. This toolkit converts the large language model into the RKLLM model format and prepares components for board-side inference.
3. Model conversion
   Use RKLLM-Toolkit to convert and quantize the model. The toolkit supports converting models from Hugging Face or GGUF formats into RKLLM so the model can be loaded and executed on Rockchip NPU.
4. Compile the runtime
   Install a cross-compilation toolchain and build the RKLLM runtime executable. The runtime implements model initialization, inference, output callbacks, and resource release.
5. Deploy the model
   Upload the compiled RKLLM model files and the executable to the board. Once deployed, the OK3588-C board can run DeepSeek-R1 locally and interact via the board debug console without network access.

02 Demonstration

DeepSeek-R1 can function as an offline intelligent assistant for a variety of tasks. Even in local deployments, it can support routine information lookup, maintenance advice for professional equipment, mathematical problem solving, and programming assistance by leveraging its data processing and knowledge base.

1. General information search
   DeepSeek-R1 can quickly retrieve and present accurate information. For example, when asked about Baoding Feiling Embedded Technology Co., Ltd., it can summarize the company's background, main businesses, and product characteristics to help users understand the organization.
2. Professional equipment troubleshooting
   For specialist equipment issues, DeepSeek-R1 can offer fault analysis and remedial steps. For instance, for a PLC reporting error code E01, the model may analyze potential causes such as power issues, wiring errors, or hardware faults and suggest corresponding troubleshooting procedures.
3. Mathematical problem solving
   DeepSeek-R1 can construct and solve equations for procurement or allocation problems, provide step-by-step verification, and return precise numeric answers and validation steps.
4. Programming assistance
   The model can generate code examples on request. For example, for UART communication on the OK3588-C development board, DeepSeek-R1 can produce a complete C example covering serial initialization, data receive, and transmit routines.

03 Performance Evaluation

After porting, comprehensive performance tests were conducted to assess DeepSeek-R1 running on the OK3588-C development board. The key observed metrics are summarized below.

• Responsiveness: Video demonstrations showed that DeepSeek-R1 returns answers clearly and smoothly without perceptible latency or stuttering.
• CPU utilization: The model runs at approximately 12%–17% CPU utilization on the OK3588-C board, indicating efficient use of processor resources for embedded environments.
• Memory usage: During the tests, DeepSeek-R1 consumed about 825 MB of RAM, allowing stable operation without memory-related performance degradation.
• NPU utilization: On the OK3588-C board, NPU load distribution showed three cores each reaching approximately 83% utilization, indicating effective offloading of neural compute workloads.

The demonstration verified the feasibility of running DeepSeek-R1 locally on an RK3588-based development board. Subsequent articles will detail the porting steps, alternative methods, and step-by-step instructions for reproducing the deployment.