Ryzen AI Max on InternetException

Running Qwen 3.5 on Strix Halo using vLLM and Docker

Sun, 26 Apr 2026 22:20:40 +0200

I have Desktop Framework for 2 months now. It is pretty capable for interefence with small contexts. With longer contexts it degrades quickly. It can for example can run 100k tokens contexts for 10minutes. Because of those limitations I decided to try vllm again - maybe it will be quicker. Two months is long time in LLM frameworks last time I was using vllm it has version 0.17. Now stable is 0.19.

vLLM is fast but kind of slow

Wed, 25 Mar 2026 18:53:05 +0100

I was using vLLM to run my AI assistant models for few weeks now. It is quite nice, production ready framework that have few bugs . ROCm support and by extent Strix Halo support is quite nice, even if AMD GPU docker image does not work on Strix Halo without aiter in custom version.

Interference speeds are quite nice. You can get 20-30t/s out of new AMD APU. Which is totally usable. Here is sample test I done on this device via LLM API Throughput Benchmark:

Running Qwen 3.5 on AMD Ryzen AI Max+ 395

Sat, 14 Mar 2026 15:09:00 +0100

I bough lately Desktop Framework with intention of running Qwen 3.5 as model for my AI assistant on Nanobot . At first I could not run this model on this hardware for some weird bug in one of the libraries. I explained why and how I fixed it here . In this post I will just put the list of packages that I used to run Qwen finally and vLLM command switches and parameters.

Assigning VRAM on AMD AI Max+ 395

Sun, 08 Mar 2026 11:33:00 +0100

Few days ago I started playing with AI assistant and I decided to buy new hardware dedicated to running LLM. I bought Framework Desktop board. So far it is really great but running more than one model is a bit difficult and I wanted to test few things while my nanobot is running undisrupted.

I did quick search on my phone before buying and people were saying that it is possible to set it up via BIOS settings up to 96GB.

Self hosting your AI assistant efficiently

Sun, 01 Mar 2026 06:27:00 +0100

I wrote previously why running models that you want to use on your daily driver is not great. Basically it is not convenient and it is fine for testing but if you want to run while doing something else it not great experience.

Of course you can have more powerful device so you will not notice few models running in the background but this is like running a fan with heater under your desk.