Local Deployment: Running Qwen 3 sizes 4B and below using qwen3-engine

Hardware & System Requirements

• CPU / Processor: Laptop class (e.g., i5-1240P or equivalent) with support for parallelization if OpenMP is enabled.
• Memory Footprint: Reduced memory usage through built-in 4-bit affine quantization executed during load time.

Installation & Launch Guide

1. Clone the repository containing the source code:

   git clone https://github.com/jakint0sh/qwen3-engine

2. Navigate to your project directory.

   cd qwen3-engine

3. Compile any necessary files ensuring you have a standard C compiler, libc, libm, and cJSON available in your environment. (Optional: Use flags for OpenMP if targeting multi-threaded performance).

   gcc -O certain_flags main.c [other_files].c -lcjson -lopenmp ...

4. Download weights directly from Hugging Face in BF16 safetensors format (the engine handles on-the-fly 4-bit quantization so no pre-processing of weight formats into special binary types like kbytes enoughs required).
5. Run the executable to access the built-in chat interface or inference loop.

   ./qwen3-engine

Optimization & Performance Tips

• Quantization Method: The engine uses simple affine 4-bit quantization during loading rather than requiring specialized offline converted binaries.
• Parallelization: If compiled with OpenMP, parallel execution can be leveraged even though current laptop benchmarks show ~1 token per second due to cache locality design choices.
• Memory/Speed Tradeoff: Prioritizes representational correctness and code clarity over maximum speed via heavy usegett's ability for ease of learning vs dense pointer math found in other implementations.

This implementation provides a highly readable and transparent way to understand transformer mechanics including KV caching without relying on complex external Python runtimes atau weights conversion pipelines.

! DYOR (Do Your Own Research)