๐๏ธ Architecture
One unified stack โ from model composition down to GPU / XPU silicon.
A unified, high-performance framework for training LLMs, VLMs, diffusion, and embodied models โ multi-backend, with native NVIDIA GPU & Kunlun XPU support
A quick tour of what sets LoongForge apart
One unified stack โ from model composition down to GPU / XPU silicon.
Measured on latest LoongForge across VLA, WAM, and VLM workloads
One codebase, two silicon stacks โ production-ready on NVIDIA GPU and Baidu Kunlun XPU
Built on the community Megatron + TransformerEngine ecosystem, with LoongForge optimizations layered on top.
XPU Plugin mechanism shields the upper stack from adaptation differences, while integrating an XPU-specific optimization toolchain.
From compact SLMs to large-scale MoE giants โ all batteries-included
Compose any ViT + any LLM backbone via a YAML file. Example โ
From install to launch โ jump straight to the tutorial for your model type
Recommended: one Docker image bundles the CUDA/XPU toolchains, patched Megatron, and TransformerEngine โ so every node trains from the same environment. Source build is also supported.
$ git clone --recurse-submodules \ https://github.com/baidu-baige/LoongForge.git $ docker build --build-arg COMPILE_ENV=hopper --build-arg ENABLE_LEROBOT=false \ -t loongforge:latest \ -f ./LoongForge/docker/Dockerfile .
$ docker build --build-arg ENABLE_LEROBOT=false \ --build-arg BASE_IMAGE=loongforge/loongforge_kunlun:py310_torch25 \ -t loongforge-kunlun:latest \ -f LoongForge/docker/Dockerfile.xpu .
Choose your model type โ each card opens a runnable, step-by-step tutorial.
Browse ready-to-run configs and example scripts to launch your first run.
Open-source models trained with LoongForge or its predecessor AIAK-Training-LLM
Next-generation fully open multimodal model โ improved data, training recipe, and scaling.
Scientific multimodal large language model for advanced reasoning.
Fully open framework for democratized multimodal training.
Domain-enhanced universal vision-language models.
Built in the open โ join discussions, report issues, and contribute
