@Zerglingman No diversity keyword found on their about, and nothing comes up with a simple search, so they look good. Still have to dig into the headers but The Cloud™ isn’t mentioned once or even referenced, so It’s doubtful. It probably exists but you have to implement it yourself which I would never do even in the niggercattle mooed for it.
@anonymous It's really simple to tell if something is running local or not: Cut off internet and see if it breaks. In the case of something like this you can also just listen to your GPU fans.
@Zerglingman I have a program that sets that fans speed to close to (or above) 50C for even mild load. But there’s cutting off the internet too. Honestly don’t think I’ll need to go that far though. While I am working with it I would need to dig into the code anyway to get an idea on how to integrate it so if it has internet headers just check for calls to the internet in otherwise benign functions.
@Zerglingman@lonelyowl Working with other people’s coding examples makes my head hurt. Not even writing code yet; just trying to wrap my brain around the concept. Still having difficulty understanding that first lesson! Oof. Hopefully it will be easier once I start to dig into the mechanisms myself.
@Zerglingman@lonelyowl Either of you work on ML yourselves? I am wondering if it’s better to prioritize getting to and defeating high-value targets (like bosses) or whether it’s better to prioritize defeating any target.
The benefits of the second I think are immediately apparent. Killing any enemy gives a virtual certainty to kill high value targets, but the drawbacks seem similarly drastic. That is that the bot will just run around killing everything and perhaps run into loops where a baddie spawns, bot kills, new baddie spawns, bot kills, old baddie spawns, etc.
The second wouldn’t face this drawback, but it’s drawback is it would be very rigid and would need multiple models for different areas, increasing the size requirements drastically.
@anonymous@lonelyowl Like you tell it what the objective is and its inputs and let it try things until it achieves it, and tell it that lower time is better, so after it tries 100 times it copies the things that gave it the 5 lowest times, or something like that.
@Zerglingman@lonelyowl I think I am starting to understand. My ultimate goal is to create “human approximating” bots and I think this is where I am getting caught up. I am trying to do too much at once at the moment.
I was thinking of working on movement first and not mindlessly walking into walls and working from there?
The biggest issue I am running into in my head is what to make as objectives. Waypoints I guess. Then do I save this training data or would that muddy the waters when I inevitably work it towards more complex goals.
Ack. I should probably stop theorizing. Too tired and I haven’t even started yet.
Yeah, that's the point of reinforcement learning. You define some simple reward function, and the algorithm will learn how to maximize it. For example, the further player will walk through level without dying, the bigger the reward, and the algorithm will decide what to do with enemies, who to kill in first place and how, or maybe it will prefer to just speedrun the whole thing without murders.
@lonelyowl@Zerglingman Can you set multiple rewards that have equal weight? Like for example gaining experience points and making it through a level the fastest way possible? I should probably just read more on the subject, but the way you put it helped everything click in my head just now. Perhaps you’ll be more insightful than an article on the subject.
@Zerglingman@lonelyowl I think I am finally understanding now. Years of linear programming have created sort of a “goal rot” in my brain I think, where everything needs to come to X conclusion. Whereas ML (Q in particular) doesn’t need me to think like this. Simply all that’s really needed from me is to give it goals and actions and let it do it’s thing. I don’t really need to think about how to get there or how it will get there.
Though in a game where revival is slow even if the bot is instantly (in game code at least) respawned, it will take quite some time to get there.
Like for example gaining experience points and making it through a level the fastest way possible?
Yep! Just keep it in mind that the algorithm will maximize the reward function, so it is possible that it will figure out that just speedrunning without harvesting any experience is still the easiest way to do so. This thing is used to find the best strategies to maximize the reward in the first place, not to mimic certain behavior. If you wish to make it mimic certain human-player like behavior, i believe it’s also possible, but will require some tinkering.
Like, making experience more or less important, or maybe changing the reward function in the middle of learning process (some dude from YT did just that when tried to make ai bot for trackmania)
@lonelyowl@Zerglingman In particular I am trying to come up with a bot that you can PvP with ultimately, which would be really rad.
I think I am simply putting too much thought into it, a problem I have had with ML since I first learned of it. Having had it explained to me in a way that I can better understand has opened my eyes to my critical error in the first place. This is not something I have to baby.
@lonelyowl@Zerglingman So I think I’ve settled into a comfortable paradigm in my head, and learned everything I can from reading though I did confuse myself again trying to read into it. I love when that happens and now it’s time to focus on trial and error. I think once I’ve got movement down the rest should come naturally especially because the code-base I’ll have to work with will be rather minimal in the end. At least that’s my expectations going into it.
On a bit of a side note, I was looking into using python code in other languages because I may need it, and figured I’d find a native language version for a different coding language. I look into it and find it uses Python wrappers. :angryglitch:
@lonelyowl@anonymous Yeah, you can write specialised stuff and it will almost always be better. Although in FPS they tend to just aimhack so "good" is debatable. Teaching a neural network is less effort on your part, and more adaptable across games.
@Zerglingman@lonelyowl Mostly the less effort part, but all the moving parts confuse me. As you can see I barely made it through “use a ML library”. Let alone figuring out how to execute proper bots in games.
I’m hoping my laziness will ultimately translate into a bot that can be used across multiple games.
@Zerglingman@lonelyowl Now that I think about it the PvP portion would be pretty hard too, depending on the game. Then add on top making it realistic.
This way could potentially solve the “bots in open world” games problem and you could actually feel like you have a living world. Hell someone some day might even implement chat AI and then you never have to play with another nigger ever again.
@anonymous@lonelyowl It's pretty trivial to make open worlds that don't suck already, with procedural generation. Which is different from *random* generation, which tends to get bland after 10 minutes.
Let me put it this way: imagine your favorite MMO. Now imagine dropping a template into a private server instead of having to waste potentially thousands of hours making a bot specific to that game that you’re unlikely to ever reuse. Now imagine this for every open world game.
Would slash the need for “realism” (which for some games is mandatory :vomit: ) in private servers. This would in turn encourage more of them.
Obviously people should demand more single player games. In the meantime, and for nostalgia’s sake, we could potentially have drop-and-train bots.
@anonymous@lonelyowl That seems pretty cool. The problem with that is that every game would interface differently, so you'd have to do up the proto/bindings separately for every game. But once you're done with that, if you've plugged them all in right, the bot should just work™, and be able to adapt to the game-specific differences within time.
Pretty interesting video related to the matter. They managed to write a pretty decent proceduraly generated universe in 22kb for Elite in 1984. There's no excuse for half assed randomly generated shit or boring empty open worlds now, 40 years later, with all the advancements in technology and processing power.
> It's pretty trivial to make open worlds that don't suck already, with procedural generation.
I rather disagree, i've heard a lot of criticism of procedural openworlds from both gamers and game designers that they start to feel empty and repetitive pretty quickly. I won't say you cant create a procedural world that will be interesting, but it's not "trivial".
@lonelyowl@anonymous It actually is trivial, it's just that most game designers are shit, especially the ones with large budgets to actually produce a 3d game. You just have to implement memory/knowledge systems, and they're not that hard.
@Zerglingman@lonelyowl Indeed. There would need to be tweaking. The hope is it would ultimately be minor.
I also feel the need to say I hate the death of private servers right as these fucking “always online” bullshit shovelware crap started to come onto the scene. There were some games that are basically lost to me now because of it, and many more I will never be able to even decide if I like them or not because I refuse to pay for this model.
@anonymous@lonelyowl Well yeah, it turns out it's actually hard. :^) Anyway, I actually suspect Wushu changed hands (internally) during 3rd inner era. Both dev and publish. Particularly after 4th inner, there was a notable drop in dev quality (translation quality was never that good, but it also got much worse; as in some lines just didn't get translated at all). But people did manage to dump the strings files and load mods so that's fixable in a private context; we can just distribute a client with fixed strings.
I'm afraid you will need to create a new model for every game, because they all have different states and different actions to predict, hence different input-output layers. But i personally haven't experimented with this kind of models as i'm more of image generator person, so i have no strong opinion on this
@Zerglingman@lonelyowl One other bonus I completely forgot to mention is the boost this would give to bots in games where the PvP is far superior to PvE (shooters and Diablo-style in particular). But this of course assumes people don’t demand single player and on top of that don’t demand single player bots.
@lonelyowl@Zerglingman Not necessarily. All you’d really need to do is change state and action values to fit the game. You may also need to change some minor operating factors so it can actually do stuff.
But again this is more of a dream and I haven’t even gotten started yet so I don’t know how feasible it is.
@Zerglingman@lonelyowl Well Diablo would be one, especially Diablo 2 where some content is locked behind multiplayer restrictions. RTS’, strategy. I could probably think of more if I had to.
@Zerglingman@lonelyowl In essence, I am suggesting what RL models already do, but for much larger and more complex games. Something no one has been willing to dip their toes into yet (far as I can tell) probably because of copyright™ concerns.
My ultimate goal I suppose is to reduce burdens on mod makers while also disincentivizing always-online bullshit. Maybe this is a bit too much power to use, again I haven’t even gotten started yet, but I still think a goal worth striving towards.
@Zerglingman@lonelyowl Well I am not sure what you mean but I think you mean these are all games you play against another person or bot? I suppose in my head I was thinking PvB = PvE. In any case games like Diablo would still benefit, and shooters as well. It could also lead the way to more realistic environments and mods in games in general.
@Zerglingman@lonelyowl I suppose I disagree. Not so much on the base premise but that it’s not always the case. For example there’s a single player to StarCraft, but the multiplayer might as well be a completely different game. However for StarCraft at least you can add bots to the game, as a stand in for players. Which at least implies both were meant for people to play by themselves at least in theory.
Just "starcraft dataset", or "starcraft player actions dataset". There are already shitload of different datasets of different stuff, i suppose it wouldn't be hard to find one for your game if it's popular.
The only stuff i've had a hard time to find is anything nsfw, on the past job we tried to create an automatic filter of "hatefull content" and there were absolutely no labeled data of corpses and blood 🥺
@supersid333@Zerglingman Yeah OpenAI is totally cucked. The model I settled on now is not owned by OpenAI as far as I can tell, but it shouldn’t be too hard to make a switch if I have to.
> train a classifier of gore on fruits > launch it in production > harvest the reports of false negatives and false positives > fine-tune it on human feedback
That’s a great plan walter, that’s fucking ingenious if i understand it correctly. it’s a swiss fucking watch! 😀
@anonymous AI/ML for bot behavior in real-time video games is expensive. The underlying operation is matrix multiplication, which is usually O(N^3) and it uses GPU resources, the combination of which should make you piss yourself.
I'd strongly recommend avoiding this if at all possible.
What if the underlying database is relatively small (lets say <100MB) and the operations rather minimal (like do thing record thing repeat). Is that still really bad?
Like no real time bot training. More like train-then-use.
@ceo_of_monoeye_dating Alright. Well I figure I’ve put in the time learning it, might as well at least try it. Since neither of us knows just how bad it will ultimately be. Shouldn’t take me very long to implement.
I have a pretty puny computer when it comes to computational power, so if it doesn’t work for me likely wouldn’t work for others.
And worse case I just get started on a traditional bot. Having access to raw game data should make my bot relatively easy to make, at least a dumb bot.
This will also let me peer behind the looking glass of ML, hopefully learn something.
All in all a good learning experience if nothing else.
@anonymous No clue - my impression is that these tools only get used for games with one such bot at most. Although it's only adjacent to my area of expertise, I'm aware that it is a thing that gets done rarely and this is why.
@ceo_of_monoeye_dating I’m not deadset really. If it is that bad of an idea, regardless of how little it needs to process at any one time, it would be far better to simply make a traditional bot. I was just wondering, given how little I expect it to need access to and how little it needs to ultimately do, if it was still a terrible idea.
Seems it is.
MDK is markov chain isn’t it? Q-Learning. I was also thinking of going observational but that’s even more intense so doubtful it’s a good idea.
There was a program for RL Q-Learning which used something called sweep learning, and supposedly it’s much faster as the expense of static resources like memory (at least that was my understanding at the time).
Any idea if any of these newer things are better or worse?