InternetException

HLinq is a library that turns a query string into a LINQ expression and runs it against your database. Version 0.14.0 is out on nuget.org . Grouping got aggregates, select got nested selects and method calls, and a value written in a query finally reaches the parameters of a database function. Here is what is new.

HLinq is a library that turns a query string into a LINQ expression and runs it against your database. Version 0.13.0 is out on nuget.org and it is the biggest release since the beginning. Here is what is new.

I migrated my blog engine to Hugo. It was running for few years with WordPress previously. It was fine most of the time, but it had a few quirks that I did not like. I was thinking to migrate it to static site generator for few years but I just did not have time to go through it. Finally I decided to give it a go with my new shiny Qwen Flash Next that helps me a lot with my more boring tasks.

I posted already about me selfhosting LLM models. Right now I have two devices:

• Framework Desktop
• Custom Ryzen Threadripper server with R9700 32GB GPU

Both are decent with llama.cpp. The biggest issue is prefill times.

I have from work subscription for Claude and Codex which I tested too for last couple of months. It is OK but lack of thinking traces really hurt to keep up with what models are doing.

I know that LLM or Large Language Models are designed to mimic what people write, but still from time to time it surprises me and make me chuckle.

I am working on small tool that runs agents in confinement that gives those programs limited set of tools that you would want them to use only. Otherwise they may try to use sudo to change system settings or kill all your docker containers instead of only few were created by them. I had several times the same problem with codex not being able to start in such sandbox because of permissions issues. I am using Qwen 3.8 for now and it thinks a lot and this particular case it was thinking about those permissions.

Qwen 3.8 is out since Friday, 17 of August. I am running it on my own server since Saturday. I am running Q4 quants from unsloth .

So far it feels like a step up from Qwen 3.6 27B, which sometimes had problems figuring stuff in my project that I am developing almost entirely using models - just as a test of how far I can push LLMs in agentic coding - mostly local models, I must add. Qwen 3.8 seems like having no problem at all.

Just couple of days ago poolside released their new model: Laguna S 2.1. I was happy to try it since it have native support for 1M tokens in context. I am doing development of hobby projects on my phone, via SSH prompting models to do stuff and 250k tokens of context using Qwen 3.6 27B usually means going through few rounds of compaction before I am happy with the results. Having longer context would be better - in theory - without those ugly interruptions and reiterating stuff to agent that were lost in previous session.

I bought new phone and I started installing applications from previous device. When I installed termux I tried to copy ssh keys and ssh config from previous installation.

This is where problems started. I could not reach any local address. Nothing in 10.x.x.x was working. No pings no ssh.

I could do that to public IPs just fine.

I tried to install new version of packages but it also did not work.

I remember when I started selfhosting my own instance of GitLab, which had simple code editor that allows you to edit code quickly inline. Then they integrated with fork of Visual Studio Code and allowed you to code in real repository directly in GL.

I was thinking of doing quick code changes to applications at this way, change some configuration of my API, quick fix of my mobile phone for home automation and etc.

I do not know… I find it funny.

After few days fighting with local LLM to force it to write documentation and comments in code, in good quality I must add, and the forcing it to write readable unit tests, after I wrote:

File is untracked (new). Build passes. Ready for review.

I just answered.

awesome mate. It is friday we deserve a brake. High five.

I was thinking for a bit and answered:

Just out of curiosity I tested Qwen3.6-27B model with MTP speculative decoding with 6 draft tokens. Whole script looked like this:

docker run --rm --name="${2:-qwen3.6-medium-vllm}" \
    --group-add=video \
    --cap-add=SYS_PTRACE \
    --security-opt seccomp=unconfined \
    --device /dev/kfd \
    --device /dev/dri \
    -v /cache/huggingface:/root/.cache/huggingface \
    -v /cache/vllm:/root/.cache/vllm \
    --env "HF_TOKEN=$3" \
    -e TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL=1 \
    -e VLLM_ROCM_USE_AITER=1 \
    -e FLASH_ATTENTION_TRITON_AMD_ENABLE=TRUE \
    -p "${1:-8000}":8000 \
    --ipc=host \
    --entrypoint '/bin/sh' \
    vllm/vllm-openai-rocm:nightly \
    -c "pip install fastokens; vllm serve Lorbus/Qwen3.6-27B-int4-AutoRound \
    --gpu-memory-utilization 0.48 \
    --dtype half \
    --optimization-level 1 \
    --enable-prefix-caching \
    --performance-mode interactivity \
    --kv-cache-dtype fp8_e4m3 \
    --language-model-only \
    --reasoning-parser qwen3 \
    --enable-auto-tool-choice \
    --tool-call-parser qwen3_coder \
    --speculative-config '{ \"method\": \"mtp\", \"num_speculative_tokens\": 2}' \
    --max-num-seqs 1 \
    --tokenizer-mode fastokens \
    --max-num-batched-tokens 262144 \
    #--max-model-len 262144"

Those are results with vLLM bench tool:

After optimizing lately my vllm startup I started optimizing performance of vLLM during inference. Thing it had abysmal performance spanning like 2-4ts. Or even 0.5ts with 180k context. That was really terrible and I started to think that I wont be able to get better performance out of Strix Halo inside my Desktop Framework.

But… I remembered people praising it and its great usability with agents and Qwen models. How is that possible since it have so terrible performance.

AI hype is calming down in some companies , but in others like in the one I am currently consulting for, we still on the hype.

Few days ago I was asked by my peer about some problem with the tool that I was mostly developing. This is fine are there are always some bugs in every software, though bug was concerning behavior on bad data and therefore out of scope really. Still I offered to help, regardless of the fact that correct fix would be to fix the bad data.

I have been running few coding agents in my spare time on my personal projects and Junie does not look great when paired with local model

First of all, local models are capable but their ability to answer prompts in sustainable speeds degrades very quickly. I was experimenting with Qwen3.6-35B-A3B and Qwen3.5-35B-A3B. Both seems to be fairly capable, which is great for a local model that can run on decent GPU that is few years old (I have Radeon 7900 XTX with 24GB of VRAM) - that is outstanding considering what we thought about computers i.e. ten years ago. If someone then would say to me that we will be able to converse with our applications in human language and instruct them to do useful stuff, I would say: ‘Impossible!’. But we can and it does work but it have limitation, mainly in context size.

vLLM has very slow time to first token (TTFT). On AMD Ryzen AI Max+ 395 it can take even 10 minutes depending on exact settings and model size.

To make my user experience of using those things myself I need to have comparison of llama.cpp and vLLM startup time and performance. But to do this I needed to perform optimizations of startup of vLLM since it is even slower to start then it was with v0.17.