Use the MLX Framework via Ollama on Mac. The qwen38-deep model requires 21GB of RAM for a 32k token context window. We recommend setting up aliases and monitoring KV Cache to prevent automatic model unloading.
Prepare Your Machine and Install via Terminal
For running Qwen3 on Mac, I chose to install via Ollama because it's currently the best tool for supporting Apple's MLX Framework. The first step is installing Ollama, then pulling the qwen38-chat model to test with basic commands. From my system checks, this model's API version responds well with HTTP 200 status, confirming that the connection between localhost and the model works perfectly. For those who need deeper reasoning capabilities, I recommend pulling the qwen38-deep model, which uses more resources but delivers more accurate results for technical tasks. Using the Terminal lets you see clear operation logs and debug issues more easily than some GUI tools that hide these details. You can download and install from the official website.
Managing VRAM and Context Window for Your Workload
Memory management on Mac with its Unified Memory architecture requires careful attention because CPU and GPU share the same memory pool. From my testing data, the qwen38-deep model has weight files totaling 18GB, which is quite substantial. However, when I tried to reserve the maximum context window of 32k tokens for large document processing, actual total memory usage surged to 21GB. The additional portion is approximately 3GB of KV Cache needed to store computation data during conversations. If you have a Mac with 16GB RAM, you may need to reduce context length to 16k or 8k to keep the system running smoothly without swapping data to SSD, which would noticeably slow down token generation speed.
Real-World Testing on Our Machine
I conducted intensive system testing on a Mac Mini M4 Pro on September 2, 2026, to verify the limits of the qwen38-deep32k-test model with a 32k context window and 3GB KV Cache. I fed in a prompt containing 20,049 tokens—a substantial amount of data. The model successfully processed it using exactly 21GB of memory as calculated. However, an interesting lesson emerged from a failure: my predictive.sh script automatically unloaded this model because it detected no production routes directly calling it at that moment, causing delays when reloading for new requests. I also encountered LiteLLM warnings about failed tool-call argument parsing, indicating that tool system configuration still needs minor adjustments to handle complex commands.
Testing Thai Language and Vision Capabilities
Beyond text conversation, I tested Qwen3's vision capabilities and Thai language understanding. Specifically, I used the Thai 5-key test set with the qwen38-chat model and found it responds well to Thai context, returning successful HTTP 200 responses. For embeddings, I tested both nomic-embed-text at 768 dimensions and embed-bge at 1024 dimensions, finding both still use the ollama/ prefix, which requires caution when calling via API. For vision features, I integrated the 18GB qwen38-fast model (MLX) with the 7.3GB qwen2.5vl model (GGUF) to enable image processing within a single workflow. Smoke testing vision routes with a 1x1 pixel PNG file succeeded, demonstrating that mixing different frameworks like MLX and GGUF on Mac is feasible while maintaining stability—though at the cost of increased local storage usage.
To follow more lab work like this, follow LINE @icafefx or download free tools at redhatai.net
