Ep 42 - AI Agents and Gaming: The Next Level of Immersive Experiences

Watch the YouTube video version above or listen to the podcast below!

Ep 42 - AI Agents and Gaming: The Next Level of Immersive Experiences Podcast and Video Transcript

[Disclaimer: This transcription was written by AI using a tool called Descript, and has not been edited for content.]

Dave Dougherty: Hello, and welcome to the latest episode of enterprising minds. We have Alex, Dave, all in attendance, so it's going to be a good show.

AI in Gaming: A New Perspective

Dave Dougherty: Today I'm going to do a shift in perspectives just a little bit to something fun and it's chat GPT or GPTs. AI for gaming. So I will readily admit off the top that this is not a use case that I have thought of.

So I'm looking forward to learning and discovering some more. So who wants to start?

Ruthi Corcoran: Yeah, kick it off a little bit. You gave a little bit of the intro, but you know, one of the things that we've been seeing happen over the last couple of decades is gaming driving a ton of the innovations and The use cases in particular for a lot of the, the technology advancements on the hardware and the software side.

Nvidia's Role in Gaming and AI

Ruthi Corcoran: So I think by now everybody's well aware that Nvidia. Has just been a powerhouse over the last decade and change within the graphics card space really to enable all of the fantastic things you can do with PC gaming. Well, lo and behold, that had a cool tangential side. capability of being able to power some of the more advanced LLMs.

So there's, there's one case in which all of a sudden you've got this whole thing going on in gaming over here, becoming a part of this current AI discussion. But now we've got these cool LLMs. We've got some advancements on the software side. What are potentially some of the feedback loops we get on the backend?

I'll throw out a couple that sort of come to mind that we can maybe tease out. And then we'll see where the conversation goes. So.

Elder Scrolls and NPC Dialogues

Ruthi Corcoran: You know, first and foremost, near and dear to my heart, are the Elder Scrolls games. I spent many hours in front of the in front of a PC and an Xbox playing Skyrim, and boy do I know the NPC lines.

Arrow to the knee, sweet roll, you name it, right? Those are wonderful. We all love them. We have many memes around them, but boy, wouldn't it be fantastic if our NPCs could truly come to life with dialogues that were a little bit more robust. What kind of active, what does that open up? Storylines and games.

What happens when all of a sudden the storylines aren't just you get to pick three, the good option, the neutral option, then the bad option, then you get, However many different storylines, what potential do we have for some of those? And then I think back to some of the fun text based games I played, you know, for a while.

I played the hitchhiker game just for fun, where you had to sort of very precisely state the different things you did within very specific parameters. Hey, maybe there's another cool opportunity. So. I will pass it off to you guys and see which one you want to run with first and let's see where it goes.

Dave Dougherty: I will be honest.

Morality in Gaming

Dave Dougherty: I, the only question that popped into my head because I'm, I'm new to this space is, does anybody really actually choose the good side story? Like, why wouldn't you just destroy the village?

Ruthi Corcoran: Literally every single time. I cannot do the bad one. Even if I try. Like, I have to go watch YouTube videos to see the bad ending because I will never ever choose the bad ending.

Ever.

Dave Dougherty: Does it, like, do you actually feel bad, like, when you

Ruthi Corcoran: have to

Dave Dougherty: destroy? Really? Okay.

Ruthi Corcoran: Yeah. True story. Okay, so

Dave Dougherty: maybe this is a Dave problem. I don't

Alex Pokorny: See, I think everybody else just saves, then completely wipes out the village, then reinstalls, and is like, yep, yep, yep, we're going to continue this conversation.

Okay. I got that out of my system. I'm good.

Dave Dougherty: Yeah.

Alex Pokorny: Yeah. I know those morality games though. It's actually kind of interesting. There's quite a few studies that show the exact same thing though, that basically majority of people all go on the positive, nice side of things. But there's also a little, I think bias with that because a lot of those games are geared toward that or they reward good behavior.

You get the extra thing, you get the whatever benefits or something like that or achievement or something. So I don't know how many games really actually play it. Neutral. I don't know. You could, I mean, that, that's actually, I mean, to Ruthie's point, this is the expansion of the LLM. I mean, you could take that narrative in a completely different direction than what they were anticipating originally with the game, and you could make it into your own little story.

Ruthi Corcoran: And I wonder what the the sort of current constraints are. Right? Perhaps you can have the dialogue capabilities. But you're not going to be able to change the constraints of the game. Like the map might still be only so big. Although I think about some of the new space exploration games and they sort of have taken the, taken the parameters off the map and it's sort of an infinitely creating thing or

Alex Pokorny: voxel based.

There's some of those which are kind of low graphic, but able to. Have large environments and still loved fairly well. I think that's going to be something

Dave Dougherty: that you'd have to have a lot of the reasoning and the logic side of it. Right. Because like, it's one thing to produce dialogue, but it's another thing to have the memory of.

The choices and the actions that were taken in the beginning and then how, how that happened, or like, what are the possible outcomes from these choices?

Alex Pokorny: And the culture, like the elements that kind of make up the, the, the atmosphere of the game.

Role-Playing Games and GPTs

Alex Pokorny: That's actually one that I've, so I've played around with some GPTs, some that are meant to be like role playing GPTs, where you can say, You know, let's start a game and I'll say, okay, do you want to create a character?

And you'd go through like a character sheet development. And then once you do that, they start you as the scenario and you kind of get into it. The other ones of the, which I found

Ruthi Corcoran: for our audience members, carrier character sheet development is a throwback to Dungeons and Dragons in which you develop a character with all kinds of personality traits or traits.

Properties, traits, skill sets, knowledge levels, and you use that character throughout your imagination gameplay.

Dave Dougherty: Thank you. I was unaware of that.

Alex Pokorny: Oh, yeah, the naming yourself. I mean, all those things, picking a name, picking gender, race. I mean, you could be an alien. You can be whatever the heck you want. So that's kind of fun.

What I found Exactly to Dave's point about the reasoning. So there are some that were developed completely for the idea of role playing. So there's like Cyberpunk 2077 one. And they use all the slang from the game. They start you off really well and kind of plays pretty well. But once you start hitting those memory limits, then it starts forgetting what kind of currency you have.

I even messed with the game a little bit of like, Oh, I got the, the payment for the end of the job and I was like, yep, I opened the, the envelope. And look at the $50 million that is in the envelope, which is supposed to be like $5,000. And it's like, yep, your $50 million, it's all there in the envelope.

It's quite heavy. , I think you're able to like, mess with the, the, the logic of it. It kind of worked for a little while, but the other ones that I found that were almost even better was a few of them that are GPTs meant to help you play a video game. Very particular for video games. So Red Dead Redemption 2 was one.

Witcher, one of the Witcher games was another one, but it was basically just supposed to give you hints and help you along with the game. And that was the whole thing. It was kind of based off a wiki, kind of a basis behind it. If you asked to do a role playing game. They were phenomenal, and they were great because they knew the side stories, they knew the quest, they knew all the characters, they knew the elements, they knew the environment, they basically knew kind of those rules that make a movie or a game fun.

And they don't try to break too many of them, and they don't allow you to break too many of them because they're like, that's not a part of this world, you can't do that, you know, if you're gonna Instead be friends with this person instead like you can't change the relationship of this person They can't become an enemy because they're your ally and that's just it like stuff like that We'll start to enforce those kind of pieces which made it a self contained adventure.

That was fun. And I think as the memory Expands, but you're still right though that if you had a custom, let's say you made your own little, you know Mario world called Dave's world or something like that. You have to make it so complete that For this thing to really be, you know, lifelike and, you know, feel like you're a part of this environment until you basically are starting to say, wait, no, you're playing Harry Potter and you're not playing Dave's game anymore.

You're just making up your own thing and now you're completely playing a different game. You're you're playing with this 1 exactly. It was like, you can't throw fireballs in 2077.

Ruthi Corcoran: Okay, so this is, this is kind of I love the way that this has moved to, cause you, you brought up a bunch of points that I would never have considered.

World-Building and Game Rules

Ruthi Corcoran: And one of the things I think about with your wiki example is what, what those provide, which it sounds like the other one didn't quite have, is the world building aspect and sort of the rules of the game within. In the world, so there's the rules of the game in terms of how you play, but then there's sort of what is permissible within the various elements of the game to make it fun because to some degree, games do need to have rules and sort of choices to be made in order to have an element of fun to them.

And one of the things that came to mind immediately when you started saying that is Magic the Gathering, which is a game that's been around for over 25 years. I think it just hit its anniversary, maybe even a little bit longer. And this game has had huge staying power. For those who don't know, it's a trading card game.

If you go into any Walmart or Target and you see that trading card section, you're going to see Magic the Gathering up there. It's been around forever. And one of the things that Magic the Gathering is based off of is this idea of you've got five different colors within this card game that you're playing, each with distinct rules and things that the colors can and can't do, which then opens up a whole bunch of creative space.

Or interactions you can have, but within those set of rules. And what it sounds like to me is when we're thinking about, you know, GPT based gaming or sort of where you expand, it still goes back to that back to basics point that Dave, you're always bringing up, like, the basics of creating a game are still the same.

It's just what capabilities you have to Bringing that game to life, get expanded.

Alex Pokorny: Two elements there. You have a creativity side of it, which is kind of creating the interesting slang that 2077 has, or the, you know, names of the spells in Harry Potter, or, you know, the types of aliens that might exist. in Star Wars.

I mean, that creativity adds the color to the game and makes it kind of interesting to play. But you're right. There's also a whole nother side of it, which is like the Game Master piece, which is how is this game going to be played? What are the do's and don'ts? What are the ways that we're going to kind of control things and push things into certain directions?

How do you create that very solid foundation that you can basically, you know, put that paint of color on top of?

Dave Dougherty: Some of this stuff thinking of something you know, you bring up Witcher or Skyrim and, you know, I've played both those games. I just got done with Diablo, the latest Diablo. I like those games because you can just like, you can go in, you can pop in, play for like half hour, you can step out. I'm not like the diehard, I'm gonna go get every You know, every trophy, every minor thing.

Berthie, the Achievement Hunter. But my thing with those games,

The Balance of Game World Size

Dave Dougherty: Witcher definitely comes to mind. How large do you actually need the world to be? You know what I mean? Like, at what point is it too big? Because if you're not, yeah, you're gonna, you're gonna miss out on the story. You're not gonna care as much anymore because you're just kind of aimlessly wandering, doing all these, you know, side quests or whatever.

And I don't, I don't know that that would be interesting, you know, for, for me at least, right? I like having

you know, like Call of Duty is nice every once in a while. I'll go from the large role playing games to then go to, like, Call of Duty because then it's, you know, this is the one stage, there are a hundred guys you have to shoot and that's it. How fast can you get through it, right? Like, that, That's a totally different experience of of gaming than these really large worlds.

That could have infinite variability.

Exploring Heavy Rain and Narrative Games

Dave Dougherty: There was one game from like 10 years ago. I don't know. I don't know that enough people found out about it, but it was called heavy rain. And it was one that just sort of ended up in my collection somehow. I think it was like a freebie for, you know, buying buying the console, but it was essentially a playable choose your own adventure mystery.

And you play this dad at the mall whose kid gets kidnapped and then you have to try to find, find the kid. And so the conversations you have with the characters impact the outcome, the choices you make, when you make them, like, do you make the choice in time? Yes or no? And then what is the choice that you made?

You know, all of that influences the outcome. So they had to write. some ridiculous number of, you know, story endings and chains and how each one crosses and, you know, where you can all end up. So that, that came to mind where it's the, you really do need the reasoning, even if you are searching for like an endpoint to the storyline with something that feels infinitely variable, you still have to have those mapped out.

And how does that actually happen?

Ruthi Corcoran: That's where I think some of the, the capability piece comes in your different games are, are going to have different purposes to your point of, like, sometimes you just, you just want to get in, kill some zombies, kill some aliens and leave. Like, that's, that's just what you're there for, right? Or I would put Candy Crush into a similar category of you're just doing a thing.

We're not worried about story building. We're not doing like, we're just doing the thing. Then you've got your other games, which are much more, Hey, let's explore. Let's collect things. Let's put things together. Yeah, a story can play to some extent, but a lot of what you're doing is more of the exploration piece.

Then you've got those more heavy narrative based games that you're, you're referencing, Dave. Um, I think Firewatch is another one that comes to mind where it's almost more of a, it's almost more of a novel come to life with, with the ability to sort of part, actively participate with choices you're making.

And I wonder if where, where, The various advancements in, in AI's comes in is opening the possibilities within each of those various buckets. Like perhaps the types of games we play don't fundamentally change, but just we get a little extra nuance and a little, it builds over time. Maybe the, maybe you get a few more mappings because all of a sudden you're not having to manually map out the different story flows as a result of those.

Thanks.

AI Agents in Gaming and Business

Dave Dougherty: So just trying to relate this to something that I might be a little more familiar with, like, as we talk about the AI sides of things at least on, on the business and the marketing side, we've talked a lot about AI agents and that's come up a lot recently with HubSpot and Salesforce having their, their conferences recently.

Ruthi Corcoran: Wait, say more on that, Dave, before you move on.

Dave Dougherty: So the idea for the agents are to have sort of autonomous AI systems, they're calling it agents because, you know, Teammate coworker, or, you know, replacer does not sit well with employees. But what they're, what they're supposed to do is like with Salesforce, they specifically have these, you know, understand and interpret.

Sort of what your customers are asking your questions using the natural language and then taking actions based on the information they receive and the information that they're trained on. So it's a very vertical piece, you know, to your point of the culture thing, like making, making the AI fit. a company culture, a marketing culture, having the lingo, having all of these things, right?

Do you think that a, that sort of AI agent piece where it might be a very small Model for a specific task could you just be running a whole bunch of those sort of smaller models within the game? Or are you thinking one, one massive piece or combination of the, of the two in order to facilitate the, the gameplay?

Ruthi Corcoran: That's such a cool idea. Some of the initial thoughts that come to mind.

Interactive Sidekicks and AI Characters

Ruthi Corcoran: As you're talking through this AI agent idea and how it might relate to games, I think of some games in which you're given like a sidekick and that, right? Or you have like a, you have a person who's following you throughout the game.

Gosh, I wish I could remember some of the, some of the names of the games. I'm drawing a blank here, though. There's one zombie apocalypse game in which there's a girl following the main character the whole time, right? I could see that being a use case and where all of a sudden you've got a much more interactive sidekick who's, who's coming along.

Maybe you can have more dialogue with them about, The thing that just happened or, or they're a little bit more sophisticated in how they're playing the game. Another thought is like, Oh, to your point about having multiples in there, that, that also opens up new space of, you know, maybe it's, maybe it's quite sophisticated to have that one sidekick in a game.

Maybe some of, some of the technology advancements make it possible. So you're going along with an entire team, each of which is semi autonomous.

Dave Dougherty: Cause at least for me with my, you know, sort of creative writing background, I'm thinking, okay, if you look at character types, so if you have like that large world, like a, like a witch or like the, the red day, a town or a village sheriff, Is a very particular personality type, even though there is a range within that, that type, right?

So, I mean, you could have like a sheriff agent that is trained on particular behaviors. They move a certain way. They have certain behavior patterns speech patterns that would be similar to other sheriff characters so that whenever they come up, you could have that combination of attributes or whatever for a particular sheriff.

The problem then would be having the memory to, you know, Sheriff Johnson is different from Sheriff, you know, whatever.

Alex Pokorny: I could make it kind of fun though. I mean, that can also kind of add some like spice of randomness into it. So let's say you're trying to rob a bank and you're just like walking around the streets waiting as you do.

And it's some little kid runs up as one does, right? And the kid runs up to the sheriff and says, Hey, I lost my horse. Can you help me? And then suddenly the sheriff runs off with the kid. And then now Sheriff's not a problem in the town and you have more access to that bank that you're trying to do. Or the other time, like, you know, someone points out saying, who's this random newcomer to the Sheriff and the Sheriff comes over to you and starts talking to you.

Like, you could, it could make it a little bit more interesting and a little bit different, but you're right. I mean, bartenders, you could have a whole bunch of different AI people. Agent bartenders who they will sell you certain things. They'll tell about the rumors of the town, but they don't sell you a horse because they're a bartender, you know, like there's limits to them.

Dave Dougherty: Isn't this what they did in Westworld? Did you guys ever watch that?

Alex Pokorny: Yeah, pretty much.

Dave Dougherty: Okay.

Alex Pokorny: Except there were like cyborgs. Well, not, but the Androids,

Dave Dougherty: well, next episode, we'll talk about

Alex Pokorny: keep taking this further. Yeah, exactly. Exactly. As I

Ruthi Corcoran: think about some of this, Oh, go Alex. And I was just a quick

Alex Pokorny: one about the thinking about the action, more first person shooter kind of games.

Waiting Mechanics and Game Immersion

Alex Pokorny: There was one, I was playing a vermin tide where basically it was an online only hand to hand combat kind of, or a first person shooter kind of a game.

So you load into the lobby, which was this bar, and in the bar you could upgrade your weapons, you could do some other stuff, and then you could talk to the bartender who just gave you like three lines. And you wait for There in this really, it's a lobby for the other players to load into the lobby before you can say, let's go.

And then you all leave the door and then you actually start to play the mission that you've selected as a team,

Dave Dougherty: but

Alex Pokorny: yeah, right, but it got really boring at times because you're waiting for people to load in and you wanted to like, play a particular map. That wasn't as popular, but you needed it for. You know, your progression or something like that, and there was nothing to do.

I mean, I've created all my weapons. I talked to the bartender, heard the three lines, and I was just waiting for other people to load in. It would be, even though it has nothing to do with the game, it would make things more interesting if the bartender had a bunch of things to talk about and you could have a conversation with them while you're waiting, basically, like it would make some of those elements seem like less.

So each place or each spaceport looks a little different on your Star Wars adventure, whatever you're doing. Or it could make it just a little bit more interesting when you're When really you're just waiting for a different game mechanic to happen, which is other players need to load into your lobby.

Ruthie, where were you going?

Feasibility and Future of AI in Games

Ruthi Corcoran: Well, I think about all the different examples we've given and the what ifs and all wouldn't this be cool. And it also just occurred to me, I have no idea. And the level of feasibility across all of these of, Oh yeah, you just plug in a GPT to the, Bartender should be easy. Like, is that true?

I have no idea like

Alex Pokorny: Skyrim to upgrade Lydia to a jpt and it kind of works and it doesn't work because there's there's limitations to it as well But the starting narrative at least with her and stuff like that. It works really well So you can play it for a little while

Ruthi Corcoran: trying

Alex Pokorny: Oh, totally. Yeah, that's pretty cool.

Ruthi Corcoran: The other thought that came to mind is the compute power needed for this and what What do the next generation of consoles need or PCs need to support all this activity or on the back end, like, what are the sort of the servers that are going to be needed? And I think back back to some of the conversation we had a few years ago about going from 4G to 5G and like how that opens up a whole bunch of capabilities that people don't even think about, but then all of a sudden it's, it's there and the way we use the technology shifts because what's possible,

Dave Dougherty: well, you're seeing that you're seeing that with the LLM pieces, like I'll have to find a link to an article on it.

But like Microsoft is trying to convince New York state to keep Three Mile Island open so it can exclusively

Alex Pokorny: did it, they did it, they're reopening a nuclear power plant just to power their

Dave Dougherty: AI. So there you go. There you go. The needs are high. You know, cue Godzilla for

Ruthi Corcoran: the PlayStation six, everybody that's

Dave Dougherty: yeah, yeah, it'll be, it'll be interesting.

And it is fascinating too, because I, I've recently, I've been in a number of conversations specifically around how to use AI, like within particular business, and they're not that much different. Then this, you know, kind of gaming scenario. I mean, it's a different, it's a different scale and a different spin on it, but really, honestly, it's, it's very, very similar.

So that'll be, that'll be fun to think about in the coming days for sure.

Dave Dougherty: So anyway, I hope all of you have found this, this interesting, like always, please like subscribe, share, and, you know, use the email that's in the description to send us your thoughts. Whether you like it or Other perspectives on, on this topic.

It's, it's pretty, it's an interesting lens to use. So without further ado, we will see you all in the next episode in two weeks.