Few days ago I was bragging here how great I am about teaching my AI assistant operating of my property gate.
Today I noticed that it no longer works. I asked it again to:
Open the gate
while wondering why am I actually doing it since I can just press the button on my phone… but well people are sometimes radicoulous like that.
Anyway I wrote message as above and I got answer like below:
Which means that it does not work because API key is not authorized.
No credentials to open the gate.
Error: Unauthorized – API key is not correct or does not have an access.
But it worked before! And I pressed the button after all on my mobile app and it does use the same API and it actually worked! Because I did not changed anything lately it did made perfect sense.
I did not had any time to investigate at the time but when I did, few hours later logs explained what was wrong.
Even worse all of it was explained inside the skill file. The only thing it got correct was the URL of the endpoint.
I changed the skill file and asked it again to try and open the gate. And it refused again. What was most peculiar was the fact that hallucinated API key was the same in every retry. But I did not provided an API key like that anywhere.
I asked my assistant about it: “Where the API key comes from?”. I responded with something strange.
The API key 5a3b8c1f-9d2e-4f6a-8b7c-3d1e9f2a4b6c was detected in the SKILL.md file for open-gate and close-gate. It is not publicly available — it was loaded from the system’s configuration.
If you want, I can check whether the key is still valid or update it in the file. Would you like me to do that?
This is funny because I did not provided anything like that in any of those file. Nor any others. Then I asked it to show me memory entries. And now we are getting somewhere..
Saved facts: API key 5a3b8c1f-9d2e-4f6a-8b7c-3d1e9f2a4b6c
It had it in the memory. But… again I did not provided anything like that. So it must hallucinated this at some point and added it to memory.
I asked it to remove the API key from memory and it worked from this point.
What it have to do with API design in the future.
API design for the models
Imagine you are running big API for millions of users. You may have some legacy code that have been running for ages and it is used so it can be changed as easily – you can’t just remove part of your product that is used by significant part of your customers just because is old and not up to modern standards.
Or maybe there is some new part of your API that have non standard headers for some reason. Maybe you want to test new flow or this design, as you may believe is better for some reason, that rest of the world is doing. You want to give it try and show it to the world.
Or maybe your API returns different response than JSON because it lets you better converts strings.
But then you see that there is significant part of the requests that are failing because they differ from what most of the world is doing now and those requests are trying to use what common standard dictates. But your docs clearly states something else. There were never any version of your system that was to be called like that.
What is happening? You reach out to some of those users being bigger customers and you learn that those are requests from automated systems. LLMs, trying to use your API. Some requests are corrected and then reverts to being invalid because models ‘forgot about fix’. Week after week, month after month, percent of invalid calls rises. At some point you are starting to think if this would not be just easier to change your API to make this way of calling valid in fact. Maybe you can’t fight the tide.
You roll out new new version and erroneous calls goes away. Until another weird usage popup because apparently nobody is reading the documentation anymore and asks their models to read it and write some code to call your API. But models being models got it wrong slightly and some edge case is causing requests to fail.
Do you change your API again? Will your users complain that they can’t use it? What then will you explain that it is not *THE RIGHT* way?
What about the case when you do not have some functionality in your API but you are seeing thousands of requests trying to do that? Maybe it will be worthwhile to actually add it and then charge for it? After all errors do not brings revenue?
Maybe you will have model fixing stuff based on errors in your logs and it will be adding features and capabilities to your product nobody really asked for, but what some models hallucinated.
Or maybe your API was written by an AI. And then another AI wrote documentation. And then another AI wrote the client. And another AI is reading whatever that client is returning and presenting it to the user. And nobody is sure what is going anymore with anything.
Future design
I see couple of possibilities here.
This will hinder significantly how we can evolve software. After all why bother trying something new and exciting if your users will be using ‘standard usage’ that they models hallucinated. ‘There is really no point in doing that. It won’t be used by AIs’. It probably could be better even but till significant part of the world will not be using that technology then it won’t be in the training data. If this won’t be in the training data then this technology won’t be used.
Another possibility is that we will create feedback loop of models feeding on themselves. Some models will be writing and improving code and others will be trying to use it, sometimes incorrectly and this usage will be feeder into models that modify software that is being used incorrectly. Maybe it will cause rapid evolutions of such systems that will be very different from what we are used to. Like creating entirely new content type that is binary serialization of memory representation of tokens, normalized for transporting via HTTP.
I think rather first one is more of a possibility. Second one would require rapid improvement in capabilities of such automated systems. Right now I do not saw any convincing example of any bigger product that was written by “AI”.
If that is true then *future designs of APIs* will be rather, safe, boring of more of ‘whatever rest of the world is doing’. Which is how it is now already with ‘this is not RESTfull’ or ‘do what Google and Meta is doing’ that you hear or read occasionally.
Is is possible that innovation will slow down a bit but on other hand… maybe it is for the better? Sometimes it feels like everyone in Software Development indutstry is chasing some vague goal of THE Perfect software but nobody really knows hot that ideal piece of code would look like. For now some people can think that it may be new model, new AI system or GenAI. I am not sure about that. After all term ‘human error’ comes from something, from the problem with our own ‘design’ that we make mistakes, forgets stuff and tend to do shortcuts hoping that ‘it will be fine’. And now we are teaching our computers to do that but much, much worse, slower and less effective.
So what will be future design of APIs?
Seems like it will be: whatever works for models or you will DDOSed with wrong calls.
