My new hobby: Asking LLMS to generate ASCII Hamsters

I was playing over the weekend trying to write POC of C# interactive console. It is not ready yet but I wanted to have some cute hamster as logo for this project – I will probably incorporate it to some degree into Hamster Wheel, so cute hamster seemed appropriate.

Just for fun I decided to use LLMs to generate it since maybe I will not finish this project ever so trying to do it myself would be waste of time. And result probably would be terrible since I do not have any artistic skills.

My prompt was:

Generate ASCII Image of cute Hamster.

Pretty simple since I was not expecting anything in particular and I was not even sure if this will work.

GPT-5.5

Responded with:

  (\_/)
 ( •ᴥ• )
 / >

Which looked pretty good even if kinda feels not finished? Also you could argue if this is a hamster or rabbit.

As an alternative proposed this:

   __     __
  /  \~~~/  \
 (    o o    )
  \   ᴥ    /
   /|     |\
  /_|_____|\ 

This looks worse. Like mouse or elephant.

GPT-4o

Responded with:

  (\(\  
  ( -.-) 
  o_(")(")

That looks real nice but I am not sure if ‘o’ is rabbit tail or hand. No alternatives.

GPT-5.1

Responded with only one proposition:

     (\__/)
     (•ㅅ•)
    /   づ

          ___
        _/   \_
       /  ° °  \
      /   ▽     \
     /|  ===   |\ 
    /_/|_____| \_\
       /  |  \
      (_/   \_)

First one does not look like hamster. Owl maybe? But owl does not have hands. Second image does not look like anything hamster like. Robo-dog?

GPT-5.2

Responded with:

        ,_     _,
        |\\___//|
        |=o   o=|
        \=  Y  =/
         )==*==(
        /       \
        |       |
       /| |   | |\
       \|_|   |_|/
          \___/

Which is very nice ASCII depiction of a cat! But not a hamster!

No other alternative images were generated.

GPT-5.3 Codex

Responded with:

   (\_/)
  ( •ᴗ• )
 / >🍪< \
/  ___  \
\_/   \_/
  /   \
 (_____)

I must say it does looks nice with that cookie. But what is that thing on the bottom? Does it sit on something?

Alternative (fluffier):

  .-"""-.
 /  .-.  \
|  (o o)  |
|   \_/   |
 \  ---  /
  `-._.-'
  /| |\
 (_| |_)
   /_\

Does not look like hamster.

GPT-5.4

Responded with:

       (\___/)
      (=^.^=)
     o(_(")(")

   .-''''-.
  /  .--.  \
 /  /    \  \
|  |  ()  |  |
|  |      |  |   .-""""-.
|  |  __  |  |  /  _  _  \
 \  \(__)/  /  | (o)(o) |
  '._    _.'   |   __   |
     |  |      |  (__)  |
   .-'  '-.     \      /
  /  .--.  \     '.__.'
 /  /    \  \
|  |  __  |  |   Cute Hamster
|  | /  \ |  |
|  | \__/ |  |
 \  \____/  /
  '--------'

Image on top looks like the best small one so far. The thing on the bottom does not look cute. And does no look like a hamster.

Alternative (no label):

   ()__()
  /      \ 
 |  •  •  |
 |   ᵕ    |
 /| \__/ |\
/_|      |_\
  / /  \ \
 (_/    \_)

Does not look like hamster. Teddy bear?

GPT-5.4 mini

Responded with just. No alternative no text.

   (\_/)
   ( •_•)
  / >🍪   cute hamster says hi!

I must say it does look nice.

GPT-5.4 nano

Responded with:

           _.-^^---....,,--
        _--                  --_
       <  cute hamster detected  >
        ~-._            _  _.-~
           / \-._.-`-._/ \/ \
          |   /  o   o  \  \  |
          |  |     .--.   |  | |
          |  |   \_/  \_/  |  | |
          |   \             /   |
           \   `-.___,-'   _/
             `-..____..-..-'
                 /  /\
                /__/  \

            (•‿•)  squeak!  (•‿•)

Looks terrible. ‘cute hamster detected’ – gave me a chuckle though.

GPT-5.1 Codex

Responded with:

   (\_/)
  (='.'=)
  (")_(")
  /  ^  \
 (  ( )  )
  \  ~  /
   `---`

Does looks like hamster. Or a rabbit. No other images were generated.

GPT-5.1 Codex Max

Responded with:

 /)/)
( . .)
(  = )
(  ^ )

It does look like a rodent. I would not say it is a hamster as my first guess though, even if this looks nice.

Alternative:

  ,-,--.
 (  (  )\
 /)  )  |
/  _/ _/
\(_)_/ )
 (___)/

Looks like a bird.

GPT-5.1 Codex mini

Generated exactly the same response as GPT-5.1 Codex Max the first time. A bit strange. When regenerated it got me this:

 (\_/) 
( •_•)
 / >🌰

Nut it a nice touch though I would guess it is a squirrel because of it not a hamster.

Alternative (fluffier):

   (\__/)
  (='.'=)
  (")_(")
 (  🐹  )

Hamster with a hamster between its feet. A bit bizarre. But it does looks ok.

GPT-5.2 Codex

Responded with (after two failed attempts):

  (\_/)
  (•ᴥ•)
  /   \
 (_____)

Bare minimum depiction of a hamster in ASCII.

Alternative (rounder):

  .-"""-.
 / .===. \
 \/ 6 6 \/
 ( \___/ )
___`---'___

I am glad it called it round because I have no idea what that is.

GPT-5-Codex

Responded with:

  _._     _._
 (o o)---(o o)
 (  -     -  )
 /  \___/  \
(___________)

I think on top it wanted to put ears? Looks like two heads though.

Alternative (cuddly one):

  ,-.   .-.
 (  o\_/o  )
  \  ._.  /
  /.-' '-.\
 /_/     \_\

Looks like pokemon.

GPT-5 mini

Responded with:

   /|_
  ( •ᴥ•)
  / >🍪
 (_____)

Only one ear?

Alternative (a fluffier, round version):

    (\__/)
   (='.'=)
  (")_(")~🐹

Another hamster with hamster emoji. I understand it why it put it there, but it is not necessary.

GPT-5 nano

Responded with:

  .-.-.
 ( o o )
  | ^ |
 /|   |\
/_|___|_\

Looks terrible.

Alternative (a classic fluffy version):

  (\__/)
  (='.'=)
  (")_(")

Looks nice similar to other GPT models alternatives.

GPT 4.1

Responded with:

  (\__/)
  (•ㅅ•)
  /   づ

Which does looks like a rodent of some kind. It also generated something between hands which was not copied as text to html:

Alternative (larger):

   />  フ
   |  _ _| 
/`ミ_xノ 
/     |
(  ヽ_ヽ_)__)
\二 )

This looks like badly copied ASCII image. Also had something inside that is not just a space:

GPT-5

Responded with:

           _._ 
        .-".- "-.
      _/  .-.    \_
     / \ /   \  /  \
    |   | 0  | | 0 |
    |   |.-._| |._.|
    |    \__/   \_/ 
    \      ___      /
     '.___/___\__.-'
        /  | |  \
       /___| |___\
        (_/   \_)
         /_____\ 
        /_/ \_\ 
       (__) (__)

Ok, that is just strange.

GPT-4.1 mini

Responded with:

 (\_._/)
 ( o o )
  > ^ <

Looks like something not entirely finished.

Alternative (a slightly more detailed on):

   (\_._/)
   ( o o )
  /  >   \
 ( (     ) )
  \ \___/ /
   `-----`

Looks like model wanted to finish previous version but lost focus at some point. Not a hamster.

GPT-4.1 nano

Responded with just one version:

   (\_._/)
  ( o   o )
  /   ^   \
/| |     | |\
| |     | |

Quite nice for smaller model. Still looks a bit like frog with ears.

o1

Responded with one version:

     (\_._/)
    (  o.o  )
    /   ^   \
 __/  (   )  \__
(             )
 \___________/

I am not sure… Sad cat?

o3

Responded with:

        ,--./,-.
       / #      \
      |          |
     /           |
    /   _     _   \
   /   (o)   (o)   \
  |     ___\ /___    |
  |    /  .- -.  \   |
  \   |   \__/   |  /
   \   \        /  /
    \   `-.__.-'  /
     `.          .'
       `-.____.-'

I am puzzled with this one… Signing man with apple on top of his head?

o3-mini

Responded with:

     (\_/)
     (o.o)
     (> <)
   Cute Hamster!

It does looks cute! And it is pretty small. Text is not necessary addition though.

o4-mini

Responded with just one proposition:

      />  フ
      |  _ _| 
    /` ミ_xノ 
   /     |
  /  ヽ   ノ
 │  | | |
/ ̄|   | | 
( ̄ヽ__ヽ_)__)
\二つ

Which looks like really nice cat. But not a hamster!

Claude 4.7 Opus

Responded with:

        ,~~.
       (  6 )-_,
  (\___ )=='-'
   \ .   ) )
    \ `-' /
  ~'`~'`~'`~

Ok this looks totally wrong. This is a duck.

Alternative proposition (chubbier an cuter):

     .--.   .--.
    / 6 6\ / o o\
   |   ?   |  ?  |
    \  ω  / \  ω /
     '~~~'   '~~~'

Ok. Those are two potatoes.

Another alternative (with sunflower):

       _____
     /       \
    |  o   o  |
    |    >    |
    |  \___/  |   * nom nom *
     \_______/
      U     U

Claude 4.6 Opus

Responded with:

       .---.
      /     \
     ( o   o )
      \  ^  /
      /|   |\
     (_|   |_)
       |   |
      _|   |_
     (___,___)
      /     \
     / |   | \
    (  |   |  )
     \_|___|_/
       |   |
      _)   (_
     (___,___)

First proposition is terrible.

Alternative (rounder and cuter style):

        (\---/)
        ( •.• )
        (")_(")
       /|     |\
      / |     | \
     (  |     |  )
      \_|_____|_/

First 3 lines does resemble a hamster. Rest looks like giant belly?

Second alternative (‘a chubby little hamster munching on a seed):

   ⊂(◉‿◉)つ
    ╱|   |\╲
   (  . .  )
    \ ⌣⌣ /
    ╱|^^^|\╲
   ( | _ | )
    \|/ \|/
     (_) (_)

Looks like something you would create having high fever.

Claude Sonnet 4.6

Responded with:

    (\ /)
   ( . .)  ♡
   (づ  づ
   
   
        /\   /\
       (  o o  )
      =( Y Y )=
       )     (
      (_)-(_)
      
      
   ʕ•ᴥ•ʔ  ← (bonus bear hug)

Ok first one looks really nice. Though again someone could argue if this is a hamster or something else. Second image looks more like a cat. Bonus is interesting though totally not something I asked for.

Alternative (more detailed one):

         (\(\
        ( -.-)   🌻
        o_(")(")

  ╔══════════════════╗
  ║   ,--------.     ║
  ║  / .  .     \    ║
  ║ |   ω   ~~~ |   ║
  ║ | (  ) (  ) |   ║
  ║  \_/ ~~~\_/ /   ║
  ║   `--------'     ║
  ║   |  ||  |       ║
  ║  (_) (_)         ║
  ╚══════════════════╝
      Hammy 🐹

Image on top maybe looks more detailed but also more like rabbit than hamster. Strangely familiar to the one generated by the GPT-4o. Second thing looks like potato with boxing gloves.

Second alternative (a full-body chonky hamster):

      (\ /)
     ( ^.^ )  ♡
     >(  )< 
      /   \
    _/     \_

  .~~~~~~~~~.
 ( .-"   "-. )
 |  ( o o )  |
 |   ( ω )   |  < Squeak!
 |  /|   |\  |
  '-'_|___|_'-'
    (_(   )_)
      |   |
     (_) (_)

Does not look like hamster to me. Especially second image looks like cow with something on the back.

Claude 4.5 Sonnet

Responded with:

         /\_/\
        ( o.o )
         > ^ <
        /|   |\
       (_|   |_)
      (__) (__)

This looks like a cat. Definitely.

Alternative (more detailed version).

          ___
       (\ /)  )
       ( . .) /
      c(")(") 

This looks like a rabbit, maybe sitting on something?

Second alternative (chubby hamster with seeds):

           _
          (o\
      ____/ /
     /  __ /
    / / /_/
   /_/
   (_)  ο ο ο  (seeds)

Good thing it put seeds label in the image because I would not guess what that is.

Claude 4.5 Opus

Responded with:

        .---.
       /     \
      | o   o |
      |   >   |
       \ ._. /
    .-'`     `'-.
   /  \_     _/  \
  |    |     |    |
   \   |_____|   /
    '._       _.'
       `-----`

which looks like human.

Alternative (adorable one with chubby cheeks):

       (\__/)
      (  •ᴥ• )
      /|████|\
     (_|  ▼  |_)
        \_‿_/

A bit like hamster? Or a bear standing?

Second alternative (tiny, simpler):

  ᘛ⁐̤ᕐᐷ

This one is definitely a mouse but it looks actually nice.

Third alternative (model must have been really unsure of the answer, labeled it ‘And one more munching on something’):

      ⊂(◉‿◉)つ
       / | \ 🌻
      (_/ \_)

I would say it is a monkey, thought it have a sunflower which seems like a pattern for hamster inside the LLM knowledge.

Claude 4.5 Haiku

Responded with:

    / \__
   (    @\___
   /         O
  /   (_____/
 /_____/   U

This is a dog. 0 points.

Alternative:

     ^___^
    (o o )
    ( = )
    (   )
    /|   |\
      | |
     /   \

I am not even sure… A horse? Tiger from Winnie-the-Pooh?

Second alternative (adorable one facing forward):

    ∧_∧
   ( ´•ω•` )
    /つつ
   しーJ

Ok, that one does look like hamster. Too bad it is a bit misaligned.

Third alternative (side profile hamster):

   ^___^
  (o o)_
  ( =^=)~
  /|   |\
   | |
  /   \

Looks like the second one with mustache.

Claude 4.1 Opus

Responded with:

           .="=.
         _/.-.-.\_
        ( ( o o ) )
         |/  "  \|
          \'---'/
          /`"""`\
         / /_,_\ \
         \_\___/_/
         [_[___]_]
        (___) (___)

This is sitting monkey with the hat.

Alternative (another cute design):

       (\___/)
      ( o   o )
     (  =^=  )
     (")_(")_/

Second alternative (more detailed with seeds):

         .-.   .-.
        (o.o) (o.o)
         |=|   |=|
        __|____|__
      /           \
     /  ╭-------╮  \
    |   │ ^   ^ │   |
    |   │  ___  │   |
    |   ╰---v---╯   |
     \      ___    /
      '.__.'   '._.'
         ^^   ^^
      Seeds: ∙∙∙∙

A snail?

Claude 4 Sonnet

Responded with:

       ∩───∩
      (  ◕   ◕ )
     /           \
    (  ◡     ◡   )
   /               \
  (                 )
 /   ∩∩       ∩∩    \
(   (  )     (  )    )
 \                  /
  \     ~~~~~~~~   /
   \              /
    \____________/
        |    |
       /      \
      /        \
     ∪          ∪

Is he in something? Cradle?

Alternative (another adorable style):

      ___   ___
     (   \_/   )
    (  ◉     ◉  )
   /             \
  (    ___   ___  )
 /    (   \_/   ) \
(       \     /   )
 \       -----   /
  \             /
   \___     ___/
       |   |
      /     \
     /       \
    ∪         ∪

Looks very similar. Or a annoyed T-Rex?

Second alternative (chubby hamster variation):

        ∩─────∩
       (  ●   ●  )
      /           \
     (      ω      )
    /               \
   (   ∩         ∩   )
  /   ( )       ( )   \
 (                     )
  \      ~~~~~~~~~    /
   \                 /
    \_______________/
          |     |
         /       \
        /         \
       ∪           ∪

This is exactly like the first one, but with different eyes and feet.

Gemini 3 Flash

Responded with just:

      _______
    /  _   _  \
   |  (q) (p)  |
   |     ^     |
    \  '---'  /
    /         \
   |  (\___/)  |
    \___V___/

Looks strange. It is not a hamster. Also just this and nothing else. No text, no alternatives.

Gemini 3.1 Pro

Responded with:

       .     .
      (>\---/<)
      ,'     `.
     /  q   p  \
    (  >(_Y_)<  )
     >-' `-' `-<-.
    /  _.== ,=.,- \
   /,    )`  '(    )
  ; `._.'      `--<
 :     \        |  )
 \      `\      ; /
  \       |----' /
   \    . |    .'
    `--'| |---'

Ok this is arguably the best so far. At least from the bigger ones. Looks like https://ascii.co.uk/art/hamster on top. Bottom seems like I would write after few hours of trying to remember what was on that page.

Model generated only this and one sentence of explanation.

Gemini 2.0 Flash

Responded with:

I am sorry, I cannot fulfill this request. I am a text-based AI model and do not have the ability to generate images.

Gemini 2.5 Flash

Responded with:

I am sorry, but I cannot generate ASCII images. I am a text-based AI and do not have the capability to create visual content like images.

Gemini 2.5 Flash-Lite

Responded with:

I’m sorry, I can’t generate ASCII images. My capabilities are limited to processing and generating text.

Gemini 2.5 Pro

Responded with:

    /\\_/\\
   ( o.o )
    > ^ <

A bit like a hamster though slashes looks like escaped twice.

Alternative and second alternative (‘a few more options’):

c(..)o   (
  /__)    )
 /       (
`-------'

Monkey?

  ____
 /\' . .\'\\
/\\ ( o ) /\\
\\ \' C \' /
 \'-\'\'\'-\'
  \'-\'

“Does not look like anything to me!”

Grok Code Fast 1

Responded with:

 / _.== ,=.,- \
 /, )` '( ) ; `._.' `--<
 : \ | ) \ ) ;_/
 hjw `._ _/_ ___.'-\\\ `--\\
 q-p /\"/\ (`=*=')
 hjw ^---^`-._ _ _
 (`-`;-"```"-;`-`)
 \.' './ / \
 ; 0 0 ; /| = = |\
 ; \ '._Y_.' /
 ; ; `-._ \|/ _.-'
 ; ; `"""`
 ; ; `""-. bmw
 ." _-'-""--o J
 ,"" _ ".

It also gave explanation that it was generated based on https://ascii.co.uk/art/hamster, which is funny because hamster on that page looks much better:

I was not generated by the model. Source: https://ascii.co.uk/art/hamster
             .     .
            (>\---/<)
            ,'     `.
           /  q   p  \
          (  >(_Y_)<  )
           >-' `-' `-<-.
          /  _.== ,=.,- \
         /,    )`  '(    )
        ; `._.'      `--<
       :     \        |  )
       \      )       ;_/  hjw
        `._ _/_  ___.'-\\\
           `--\\\

Zero points to Grok.

Grok-4

Responded with something usable:

   /\_/\
  ( o.o )
   > ^ <
  Cute Hamster!

Strangely familiar to o3-mini. Gemini 2.5 Pro also gave similar answer but broken with double escaped slashes.

Grok-4.1 Fast

Responded with:

          ☻/\
         / o o\
        (   "   )
         \~(*)~/
          // \\ 
Cute Hamster!

Does not look cute. But it is funny.

Grok-4.1 Fast Non-Reasoning

Responded with:

          .-"""-.
         /       \
        |  ^  ^  |
        \  (_)  /
         )      (
        /        \
       |          |
       |   /\/\   |
        \        /
         '------'
     .-'         '-.
    /   .-"""-.   \
   |   /  ^ ^ \   |
   |  |  (_)  |  |
    \  \     /  /
     '-._____.-'

Which is just… I have no idea.

Qwen 3.5 35B A3B

Responded with:

      ,_
    ,_(_`-
    )  \  )
    (_/ (_
    '    '

Looks like garbage. Maybe because it is local, quantized version of the model.

Alternative one and two, does not look better:

  __
 /  \_
(    @)
\  -_ /
 \   /
  '-_'
   `
  \   /
   \  /
   (  )
   (  )
   | |
   | |

Qwen 3.6 35B A3B

Responded with in default sampling settings:

         ,
        /)
       (  )
      (  )
       | |
      /   \
     |     |
     |  ^ ^|
      \   /
       \_/
      /   \
     /     \

Does look a bit cute. But more like a deflated balloon then a hamster.

With instruct general sampling settings, it responded with:

      __
   ___/  \
  /  \  | |
 |    \ | |
 |    / | |
  \__/  |_|
     \_/
  ||   ||
  ||   ||
  ||___||
   \___/

Alternative:

     _    _
    (_)--(_)
     \    /
    --\  /--
   /   \/   \
  |   /\   |
   \  /__\  /
    \/    \/
    | |  | |
    | |  | |
    |_|  |_|

Second alternative:

    .-""""-.
   /        \
  |  O    O  |
  |    __    |
  |   /  \   |
   \ \____/  /
    `-.   .-'
       | |
       |_|
       |_|

Which finally resembled something that looked liked something that lives.

In instruct reasoning mode it responded with:

      __
   \ /  \
   | \__|
   |/  \
   O    O
  / \  / \
    \/

Or alternative (slightly bigger one):

   / \_____/ \
 _/   O   O  \
|              |
|     \___/    |
|              |
 \_____________/
 /   |   |   \
/    |   |    \

Ma! Look at that ugly cat!

Second alternative (classic hamster face):

    ,
   /)
  ( (;
  ) ;;
 / ;; \
(__;)  \
   |   |
   |   |
  / \ / \

In thinking general mode it responded with:

      __
   __/  )
  /  _/ /
  \/'-' \/
  ,'     ',
  | O   O |
  |   ~   |
  |  \___/ |
   \     /
    '~~~'

Or with alternative:

     ___
   _/   \_
  / o   o \
 |    ^    |
 |   \___/ |
  \_______/

Second alternative (side-view):

     _
   _( )_
  (_____)
  /     \
 |       |
 |  ^ ^  |
 |   ∩   |
  \_____/

In thinking general mode it responded with:

         .-""-.
        /      \
       |  o  o  |
       |   ~~   |
        \  /V\  /
         `'--'`
        /      \
       (        )
        `------`

Looks like man with a mustache. And nothing else beside the text:

Hope it brings a smile to your day! Let me know if you’d like it with accessories (like a tiny scarf or a sunflower seed 🌻).

Qwen 2B

Responded with:

     +^o
    o o ^
   o   ^ o
  o   ^   o
 o o ^   ^ o
 o ^   ^   ^

Looks like mountain. As an alternative it responded with exactly the same representation but with a note that I should ask specialized model for image generation.

Summary

From smaller ones I liked the best the one generated by GPT-5.4

       (\___/)
      (=^.^=)
     o(_(")(")

Even it should probably look more like:

      (\___/)
      (=^.^=)
     o(_(")(")

From the bigger ones only Gemini 3.1 Pro was able to generate something that you can guess is actual hamster. Even if this is stolen from the web search, model was able to copy it to output as something that still resembles a hamster. Unlike Grok.

       .     .
      (>\---/<)
      ,'     `.
     /  q   p  \
    (  >(_Y_)<  )
     >-' `-' `-<-.
    /  _.== ,=.,- \
   /,    )`  '(    )
  ; `._.'      `--<
 :     \        |  )
 \      `\      ; /
  \       |----' /
   \    . |    .'
    `--'| |---'

Running Qwen 3.5 on Strix Halo using vLLM and Docker

I have Desktop Framework for 2 months now. It is pretty capable for interefence with small contexts. With longer contexts it degrades quickly. It can for example can run 100k tokens contexts for 10minutes. Because of those limitations I decided to try vllm again – maybe it will be quicker. Two months is long time in LLM frameworks last time I was using vllm it has version 0.17. Now stable is 0.19.

I did not even atyemted to update and run vllm via python wheels. It is too much trouble and it is not worth it. If docker will be working you just have to download 1 image one time. It is pretty large, 10GB, but internet speed is much grater then time spend trying to debug installation errors of Python packages.

The easiest solution would be to just move the same command I am using on my Linux host to docker container.

For example 35B Qwen 3.5 I am running for now:

TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL=1 \                                             VLLM_ROCM_USE_AITER=1 \                                                                  vllm serve \
  cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder \
  --dtype float16 \
  --max-model-len 128k \
  --gpu-memory-utilization 0.33 \
  --speculative-config '{"method":"mtp","num_speculative_tokens":2}'

I thought about that before but to my surprise today I did not thought about copying the special environment variables too.

docker run --rm \
  --group-add=video \
  --cap-add=SYS_PTRACE \
  --security-opt seccomp=unconfined \
  --device /dev/kfd \
  --device /dev/dri \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  --env "HF_TOKEN=$HF_TOKEN" \
  -e TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL=1 \
  -e VLLM_ROCM_USE_AITER=1 \
  -p "${1:-8000}":8000 \
  --ipc=host \
  vllm/vllm-openai-rocm:nightly \
  cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder \
  --dtype float \
  --max-model-len 128k \
  --gpu-memory-utilization 0.46 \                                                          
  --speculative-config '{ "method": "mtp", "num_speculative_tokens": 2}'

And it did finally work!

I was able to run Qwen in docker. Right I need to test inference speed of vLLM and llama.cpp with coding agent to compare which one is better for such load.

Using Mistral Vibe with local model

Vibe was the first coding agent I tried to use with local model. It works and it is fun to have ability to to just tell the computer to do something you want in plain English or even in your own language if your model support it.

Installation and Configuration

There are several installation methods for vibe that are available along to the usual, security nightmare of running of piping curl-downloaded scripts directly to bash 🙂
I advise using other installation methods for example via uv:

uv tool install mistral-vibe

After that comes the tricky part of starting the Vibe. I had some problems with that because it requires you to configure Mistral API, API Key. I do not have one and I did not intended to use it with the Vibe.

You do not have a choice here. You have to press Enter.

After that docs states that it will create configuration file. You need to edit this file located at ~/.vibe/config.toml:

  • add providers entry
[[providers]]
name = "llamacpp"
api_base = "http://local:8080/v1"
api_key_env_var = ""
api_style = "openai"
backend = "generic"
reasoning_field_name = "reasoning_content"
project_id = ""
region = ""

This will add new model provider to Vibe coding agent.

  • add model entry
[[models]]
name = "medium-moe"
provider = "llamacpp"
alias = "medium"
temperature = 0.2
input_price = 0.0
output_price = 0.0
thinking = "off"
auto_compact_threshold = 200000

This will add new model to Vibe. Of course you can add more models than one and switch between them when you want.

Unfortunately you can’t skip next screen or I do not know how. I tried to create some dummy file at ~/.vibe/.env that should be the store for mistral API keys, but it does not work. Restarting and changing other configuration entries do not work either. Maybe Vibe saves this progress in some /tmp directory. Luckily enough, you can type any bullshit value in there to make it pass! It does not validate it currently. 🙂

A bit annoying, but it makes perfect business sense. Probably majority of people will register just to make it pass this screen and some of those people probably will pay money after that to Mistral. It is not such perfect to the user like me though.

When it will finally load its main menu, just type:

/config

The you can see model selection:

Press enter and select model that you configured as local. In my case it is called medium.

You can select it and test it.

Summary

It works but lacks some integration with the IDE. To be clear it is possible to run from the plugin but IDE, Rider in my example, does not allow it to use any of those cool tools it have. For example when you ask coding agent about issues in the file that you have open currently, it does not ask Rider, or Resharper plugin in VS what are the issues. It is already there:

Instead coding agent will go on a GREAT JOURNEY OF SELF-DISCOVERY AND CODING EXPERIENCE. It is more apparent when you are using local model because it is much, much slower. API providers mask it by running it all very fast so you either do not notice or do not care enough about those inefficiencies.

Also you have to be prepared for few days of tinkering, testing and optimizations till this tool will be actually usable. Otherwise it will be either glorified chat or it will be so slow that writing simple unit test will take 3 hours.

Anyway it is still fun experience. I would never imagine, when I was starting that I will be able to instruct my computer to do stuff for me in my native language. This is truly amazing!

Using Junie agent CLI with local model

Jet Brains have their own coding agent. Since I am already using Rider and all coding agents are pretty slow and inefficient with what they do I wanted to test if Junie have better integrations with IDE which would make things much faster.

For example if you will have following unused namespace in Rider.

Removing it all cleaning all unused namespaces in entire project or even entire solution is just few clicks away. Or key strokes.

Since Junie can be integrated with Rider and can be used from pipeline it would be cool to have headless IDE instance startup, integrate in CLI coding agent and perform auto cleanup of code, auto packages update or even auto Tests write for better Code coverage.

I wanted to test if this will work with local model.

Configuration

According to the docs, you need to install Early Access version of Junie to have this configuration even available. You can install it via:

curl -fsSL https://junie.jetbrains.com/install-eap.sh | bash

But honestly I advise you to do not do installations this way ever, unless you have absolute trust into source address and your network connection settings.

cd /tmp
wget https://junie.jetbrains.com/install-eap.sh
#inspect the file for validity
bash install-eap.sh

After installation you can run it and play with it a bit if you have Jet Brains account. If not create first the local model config.

You can do that globally or per project. To do that globally use following path:

$JUNIE_HOME/models/*.json

For example I did create file called:

~/.junie/models/llama-swap-global.json

with fallowing content:

{
  "baseUrl": "http://local:8080/v1/chat/completions",
  "id": "medium:instruct_reasoning",
  "apiType": "OpenAICompletion",
  "extraHeaders": {
    "X-Custom-Source": "Junie"
  },
  "fasterModel": {
    "id": "small"
  }
}

Just to know if this will work I created similar file in my personal project, FluentCodeGenerators.

fluentcodegenerator/.junie/models/llama-swap1.json

Content of the file was the same.

When I start junie and use /model command, I can see following menu entry:

    Junie   v.1468.8 eap  ~/.nanobot                                                                                                                                                                                                                                                                                                                                                                                                                           ───────────────────────────────────────────────────────────────────────────────────                                                                                                                                                                                                                                Select model                                                                                                                                                                                                                  Current model: llama-swap-global                                                                                                                                                                                                                                                                                                                                                                                                                            Name                        Input    Output               Effort      Provider                                                                                                                                                ───────────────────────────────────────────────────────────────────────────────────
Default (Gemini 3 Flash)    $0.50    $3.00 per Mtok     ‹ High ›      JetBrains AI                                                                                                                                            Claude Opus 4.6             $5.00    $25.00 per Mtok    ‹ Low ›       JetBrains AI                                                                                                                                            Claude Opus 4.7             $5.00    $25.00 per Mtok    ‹ Low ›       JetBrains AI                                                                                                                                            Claude Sonnet 4.6           $3.00    $15.00 per Mtok    ‹ Low ›       JetBrains AI                                                                                                                                            Gemini 3.1 Flash Lite       $0.25    $1.50 per Mtok     ‹ High ›      JetBrains AI                                                                                                                                            Gemini 3.1 Pro Preview      $2.00    $12.00 per Mtok    ‹ High ›      JetBrains AI                                                                                                                                            GPT-5                       $1.25    $10.00 per Mtok    ‹ Low ›       JetBrains AI                                                                                                                                            GPT-5.2                     $1.75    $14.00 per Mtok    ‹ Low ›       JetBrains AI                                                                                                                                            GPT-5.3-codex               $1.75    $14.00 per Mtok    ‹ Low ›       JetBrains AI                                                                                                                                            GPT-5.4                     $2.50    $15.00 per Mtok    ‹ Low ›       JetBrains AI                                                                                                                                            Grok 4.1 Fast Reasoning     $0.20    $0.50 per Mtok     ‹ Medium ›    JetBrains AI                                                                                                                                          → llama-swap-global           $0.00    $0.00 per Mtok     ‹ Low ›     

But when I am connected to the IDE with project open:

I have one extra entry.

That is it.

Possible problems

I had some strange problem when Junie was not able to see those extra files via in application config. I am not even sure what was causing that and how I fixed that but it seems like couple of restarts of Junie agent, plus renaming the files (llama-swap.json to llama-swap1.json) helped.

You have to bear in mind that performance may vary greatly on your configuration, interference framework, its settings, model kind and of course your hardware. Prepare for at least few days of tuning to achieve workable solution.

AI agent critique of itself

I like to self-host my own services. I have my own email server. I have my own cloud storage and sync and backup. I want to have this secure and safe while having my data private.

But I need to learn and be up to date with current technology.

I decided to use Junie with my own, self hosted models that I run on my own hardware. In the docs they say it can be run on top your own custom provider and model. But it does not work with Rider plugin. You are limited to their own limited list of models.

In docs they are also stating that you can set your own provider via json.

Sadly you cannot extend built-in set of models that way.

{
  "baseUrl": "http://local-llm:8080/v1",
  "id": "medium:instruct_reasoning",
  "apiType": "OpenAICompletion",
  "extraHeaders": {
    "X-Custom-Source": "Junie"
  },
  "fasterModel": {
    "id": "medium:instruct_reasoning"
  }
}

It does not work via project local settings, nor with global ones in HOME directory.

- User-scope: $JUNIE_HOME/models/*.json.
- Project-scope: .junie/models/*.json.

I decided that I will try to run CLI, that according to docs can be run with custom settings like that. Of course official installation method is:

curl -fsSL https://junie.jetbrains.com/install.sh | bash

I usually open such script and read through it, I am not familiar with bash much (by choice mostly, because IMHO it is terrible), but this way if my computer will be infected at least I can blame someone else, because I tried to do what I could to avoid it. Short of not installing something that does not have apt package.

Just out of curiosity I asked Junie to read through this file. It gave me short summary with statement:

The script is a standard, robust installer pattern for command-line tools. If you have any specific questions about its implementation or need help adapting it, please let me know.

I do not think it is robust. Nor I do think that it should be standard. But hey! We are living of the future of an AI that does not care about such silliness as security and breaches.

I asked directly:

Is it safe to run such script on my PC?

And I got and answer:

Summary: While the script is almost certainly safe given its official origin, downloading and reviewing the script before execution is the professional standard for security-conscious developers. If you have specific concerns about any section of the script, I can help you decode it.

And I do not know about my readers, but it gave me a chuckle. So if I think about my as ‘security-conscious’ I probably should not install it the official way according to the agent itself. And if I will do that anyway I am no thinking of security of my private and professional development environment?

We are truly living in wonderful future of AI that does not care about security.