(Sorry, couldn’t come up with a sillier title)
So, I’ve been kind of silent on SCHNAIL recently (Starcraft Human ‘N’ AI League). This had a multitude of reasons, not all of them public, and precisely zero of them important. The thing that is important however, that I’m back on track building it. If this is the first time you hear about it, you might want to read up on the origins here and here.
The original idea was to have something that creates games over the network. We set up a remote game server, a coordinator, and a database to keep track all of it. It worked fine, but I realized this is just overkill. This would make sense, if this was a competitive affair, where you can get unfair advantage by modifying something. At this point, there is no real benefit having this whole ecosystem. If you want to cheat against bots, be my guest, you will get a lesser amount of good boy points or something.
So I decided to keep it a bit simpler. Bots now will be downloaded to the user’s local computer (they don’t occupy that much space, all of the enabled bots on the SSCAIT ladder currently take up 747 MB space on my hard drive). This will be a bit easier on resources, and not dependent on the servers being online. Also, less server costs from our side. Although there will be some, as I can’t rely on SSCAIT resources – and I can imagine that some people will want to write bots that are tuned especially for man vs. machine play.
And before diving in, let’s talk about the ultralisk in the room – Why not Remastered?
First of all, because I can’t. SCHNAIL is built on the Brood War API (BWAPI), which – at the time of writing this – is at version 4.4, which only supports 1.16. This might change in the future, but I’m not one of the great folks who develop BWAPI, so I have no pull in this. If and when it comes, I will be the first to adapt the client for it. But I’m guessing that many bot authors will be a bit slow to do so, and 1.16 support will have some momentum, and hey, people might just want to play some of the old bots as well. Eventually, when we are in the middle of it, there will be a time when 1.16 support will be deprecated. But this is not that day.
Also, I think if there is interest in playing against bots – even on 1.16, that will help showing that yes, there is interest in this. Usually when I talk about my passion to people, their first reaction is “Okay, how do we play against this bots?”.
From my experience, writing a halfway decent bot that can beat the stock AI can take weeks, if you are starting from scratch. Although we kind of did in 3 hours with NiteKat, on stream, that is more of an exception than a rule – we are both experienced in bot development, and NiteKat works on BWAPI itself.
Who is the intended audience of this? Well, I spoke/wrote about this in detail before, but let me reiterate. There are four distinct groups I would like to offer this.
Bot authors: First, and most important, but not most numerous group. I’ve been in contact with many of them, and generally been trying to get exposure to this scene. (I’d hesitate to call it a hobby at this point – the technical expertise needed to develop a somewhat good bot is on par with experience needed for an actual job. In fact I know multiple people who are getting paid to develop Brood War bots.) Many of them wants to test their bot against actual humans too.
Casual Gamers: So Brood War have somewhat limited replayability when it comes to beating AI. Not like I haven’t played the campaigns at least 10 times in my life – and the modding community has given us some nice custom ones as well – but still, maybe I want a challenge. And I really, really want to avoid the experience of coming home from work, jumping on bnet, then enduring attempted verbal abuse from de4thkill0rXXX, the most edgy 12 year old on the planet. So, you can try yourself out versus these bots stress-free.
AI researchers: This kind of overlaps with the first group (Looking at you Dave). I believe the data generated by these games can be an invaluable research tool. Basically crowd-sourcing building your dataset, when you do something with machine learning. I have a simple rule about SCHNAIL: Your data belongs to you. That means that all the data generated by the bot can be used by the author for research purposes, and likewise, the player can access the data generated by the client. Currently this just means replays, but I do plan to add some gameplay statistics collection, likely on an opt-in basis.
(And if you try to abuse the player’s personal data, I will come after you with a poisoned adamantium chainsword)
Professionals, and people trying to get a competitive edge: When you are trying to get a higher ranking on the ladder, be a better player, finding a practice partner can be hard. Even if you manage to convince your neighbor Hans to spiele das Sternenhandwerk1, he might get bored after a while, and certainly won’t be too consistent. Meanwhile, a bot doesn’t mind doing the exact same thing always, and you can get a pretty good practice versus a certain tactic, or matchup. Currently this field is sort of unexplored, but it is one of the areas where I see great potential. I want to be there when the first programmer gets hired by a pro SC team to develop practice bot. (I kinda want to be the guy, to be honest)
And now, back to the war stories, and technical problems!
So first of all, what I had from the online SCHNAIL client, was the user’s part, let’s call it client side. This was just a simple SC instance, with BWAPI injected. Currently it is only used for the automenu functionality, which might seem overkill, and by itself, it is. However, one of the long-term planned feature is to inject an actual bot to the client process, that collects statistics, and monitors the player. The client process was working nicely, but I had to figure out how to have a bot as an opponent.
First, I started examining DropLauncher code. DropLauncher is the old and trusted tool for man vs. machine matches, except it is not updated anymore, and half of the bots are not working with it. Also, setting it up is a little bit cumbersome. I’m not saying this to hate on it, the code itself helped me a lot, I just have the impression that this is wasn’t really built with gamers in mind.
There are two categories of bots that are currently supported: Module (DLL) bots, and client bots (jar and exe files). The dll is injected by BWAPI itself, and the client files you have to run separately.
First things first, I definitely needed a second instance of SC. But there is no sense launching a whole new SC with graphics and all. Also, alt-tabbing and such will be an issue, and you could see the enemy bot’s play, which I think is not really fair to those gentlebots. Also, unnecessary drain on system resources. Remember, Brood War doesn’t use GPU. Brood War predates modern GPUs.
The answer to this conundrum is using bwheadless. It is exactly what it sounds like, a Brood War launcher without graphical output. All the StarCraft, none the fun. I set it up (figuring out the proper arguments was somewhat of a hassle, so I just looked at DropLauncher code), and tried to connect an UDP game I created, but it has failed. After figuring out that I need to redirect the output of bwheadless, I finally got some logs.
They were not helping or anything, but I had them. I tried launching the client SC, and bwheadless in varying orders, and I got an error saying “Unable to initialize network provider”.
Okay, I can work with this. I contacted the relevant parties for some professional advice.
Bearing that in mind, I found out by trial and error that I need to set adjust the valueshared_memory = OFF in bwapi.ini. This resulted in games sometimes starting, but the bots were not really doing anything. In the end, we figured out that the UDP LAN option is not what we need. The bwapi team has thought of this before, and there is something called DirectIP which adds another option to the StarCraft LAN menu, and allows multiple StarCrafts to connect to the same game on the same PC. At this point, the hack count was over 9000 steadily increasing (BWAPI is a hack of BW, but you might argue that it is a part of OpenBW, which is a reverse-engineered hack of BW, DirectIP is a further alteration of that).
Anyway, let’s try this out finally. At this point, logging was yet again lost, because I wasn’t launching the bwheadless process separately.
It was time to bring in Swedish reinforcements yet again. This time, I knew how to convince them.
The logging itself was quickly solved, I just used separate threads to do it. Which of course had their own quirks, but at this point I just logged to stdout, so who care. 2 .
Everything seemed fine, but I was getting messages that the bot is not loading, with all the info being :: ERROR: Failed to load the AI Module "AI/BINARY_AIMODULE.DLL". Double-checking the paths didn’t help. It was time to gather some testing data. At this point we had around a 100 bots, majority of them being DLL bots, so naturally, I tried out all of them. And sure enough, the second or third one actually worked. This was definite progress! I noted down all the bad boiz that did not work, and their version. Fortunately, that was logged at least.
Some versions clearly did stick out, especially 3.7.5 and 4.1.2.4708. But there were some bots that were giving errors, and did not work, or were working, but throwing errors about not being able to read one file or the other.
Yet again, my eyes gazed upon the source of DropLauncher. Turns out it provides a BWTA (terrain analysis library) cache for bots, and some of them were missing just that.
Other than that, in the SSCAIT environment, bots have access to a /read folder. After making sure that the processes were started in administrator mode, this was finally working too – although I just copied everything into the /ai folder, so this won’t be a problem. And after much frustration, my efforts bore fruit.
Sometimes the bots were injected to the wrong instance. I started a game, and saw the bot playing instead of me. It was some next-gen Solitaire level bullshit. This was mostly due config issues, but I noticed that some bots were joining the first SC instance they could find. This was smoothed out for DLL bots, but later it will come back.
Then it was time to move onto the clienty bois JAR and EXE bots. First I tackled the EXE ones, and launching the user SC, bwheadless, then the .exe worked for the most part. Less so with JAR bots, many of them just froze. This was the point where I put each process into it’s own thread, so I could get logging.
Turns out I can’t really tell the bots explicitly which instance to join (I specified every parameter I could), and they tend to misbehave, and join the first one they like. So the obvious solution would be to launch the bwheadless instance first, and then the player’s one.
Except that didn’t really work, because the actual point I need to start the client is when bwheadless is searching for a game to join. Now, how can I tell this? I looked through bwapi code, I talked with people, but turns out I don’t really have a hook into this, or something that can tell me that this is happening, except for the log message that bwheadless is searching games to join.
Except for the log message. Okay, in a “this might be crazy enough to work” moment, I created this piece of extraordinary software engineering.
while (!bwHeadlessLogBuider.toString().contains("searching for games to join")) {
}
System.out.println("Bwheadless launched, starting client...");And… it worked perfectly. I was dumbfounded, but well…
Almost all the Java bots were working. The ones that weren’t tended to be older bots with the archaic JAVA_JNI interface. All I needed this point was some polish and user-friendliness. For example, the bot and SC processes were not cleaned up sometimes, and proper error handling was missing for in a lot of places. But I tried out with a standalone installer, and it basically works out of the box! I was reasonably happy at this point, and I never do that.
This concludes the technical rundown for now. It’s been a ride, and it’s far from over. I need to set up my own servers, website, and upload functionality, but all of these are just busywork. I know how to do them, there are no real technical challenges, all I need is time.
And now, a video of the current state:
And for the title: The project is in (closed) alpha now, and I had some friends, acquintances, and streamers try it out. Expect some streams soon – I will post about them on my Twitter, and Facebook, and will try to get recordings as well. And finally, some closing thoughts.
When it will go public and how can you help
I do most of the development work on it by myself (I have help of course), so I can’t commit to exact dates, but the broad schedule is to have an open beta in January 2020. If you want to help me out with development, you’re welcome to do so, but I don’t have high hopes about that.
I encourage you to subscribe to the mailing list of this site – usually there are only mails of new articles, but in this case, I will make an announcement about this, so you can join the closed beta. I’d like to launch as soon as possible.
In the meantime, well, supporting on Patreon works. Servers cost money, and I personally took a liking of not starving, so the less I have to work on my day job, the faster this will get done.
How this will affect my other projects, like the Undermind, and the book?
Sleep is overrated anyway.