Captions provided by White Coat Captioning (https://whitecoatcaptioning.com/). Communication Access Realtime Translation (CART) is provided in order to facilitate communication accessibility and may not be a totally verbatim record of the proceedings.

JASON LENGSTORF: Hello, everyone. And welcome to another episode of Learn With Jason. Today, we're going to be digging into something I've been kind of avoiding. That is, digging into the world of AI agents. I feel like we've just about everything described as an AI agent these day and the volume of discussion about it has kind of made me tired, but we're going to try to dig through that and sort of demystify what the heck is an agent, why should we care, what do they allow us to do and to do that, we are going to be talking to one of my favorite people, Dr.�Sarah Guthals. Sarah, how are you doing?

SARAH GUTHAIS: Doing good. The sun is shining superbright in Washington right now.

JASON LENGSTORF: As a fellow Pacific Northwesterner, we are solarpowered. I've gotten out of sweatpants.

SARAH GUTHAIS: I put on a nice shirt today. [Laughter].

JASON LENGSTORF: For folks who aren't familiar with you and your work, you want to give us a bit of a background?

SARAH GUTHAIS: I have a wild ride of a career. I have all my degrees in computer science, but I focused on education. I focus on how can I bring together teaching other people how to leverage technology� and I say "technology," because it could be frameworks, tools, whatever� to make something amazing. Whether that's teaching kids how to code, whether that's being an engineering manager or build tools for other engineers to leverage, whether that's DevRel, that's where my intersection and passion lie.

JASON LENGSTORF: Love it. I mentioned this a little bit before I brought you up on stage, I've been resistant to the idea of digging into agents and the a lot of the technology in AI, in general. One of the things I try to avoid is this breathless pursuit of whatever the brandnewest thing is. I want to make sure it has some utility and the pace of some of this AI stuff has been so fast that a little bit of it feels like, I'm going to learn this and it's not going to be valid in three weeks. One of the words that has been sticky enough that I feel like it's time to dig into it a little bit is this idea of a AI agent. Every time I start digging into what an AI agent is, it feels like it's a magical word that describes everything that's happening in AI right now. Could you maybe start by helping us demystify, what is an AI agent?

SARAH GUTHAIS: I kind of even want to take it a step higher, if you'll allow me.

JASON LENGSTORF: Please do.

SARAH GUTHAIS: So I've been lucky enough, where I started my computer science education journey before code.org. I was teaching kids how to code, just, like, on the weekends while I was doing grad school. I was teaching teachers, and kids, how to use Minecraft to learn how to either code or just develop those problemsolving skills that are useful within this industry, before Microsoft acquired Minecraft, right. I'm not saying I'm a hipster� I'm not saying it. [Laughter].

JASON LENGSTORF: We're saying it, don't worry. [Laughter].

SARAH GUTHAIS: But the fact that I use the word, "hipster," probably shows that I'm not. [Laughter]. It's okay. I'm a nerd. But the reason I loved being in that space, at the time, was because one of the very first papers I wrote was, um, computing as the fourth R. Why every single person should know reading, writing, math and some programming. Not because everyone should be a developer, but because in today's, tomorrow's world, everyone will use technology in some way within their industry. If you go to farms nowadays, the IoT is amazing. I was teaching foster students and one girl was like, I don't care about any of this. I was like, that's fine. What do you want to do? I want to be a preschool teacher. What if some of your students don't know how to read and some of them are more advanced? Maybe you can make a very simple appreciate you sharing for the advanced students to do on their own where they can click on a word that don't know and it will say it out loud and she was like, oh, that's amazing.
When code.org started, there was discourse around, does everyone need to know how to code? Now we accept it a little bit.
I got to work at Microsoft before the big boom. It was like, do computer scientists, do programmers, do developers really need to learn about this machine learning thing? Yeah, it could be cool for the advanced stuff. The every day engineer, do we care? No. This new world of AI, I feel similarly in the sense of, like, it's not about the fad. It's not about the excitement and the buzzwords for me. For me, what it's about� what I love to see is another person who has a passion, who wants to build something, and unlocking other possibilities for how they could help whatever part of human life they're looking to help.
So, yeah, we're going to get the, like, I don't know, capitalistic ones, right. Of course we are. But, as an example, there's actually� my grandfather has really severe dementia and I'm responsible for my mom, my daughter and my grandfather. So my mom helps my grandfather a ton. He forgets where he is all the time. He forgets who she is. He lives in her room. We don't have any outside help, so it's a lot of work for her and when I came across this, in particular, LiveKit and Twitter, what inspired this originally was, imagine if, first of all, the AI voice could not sound so robotic, right. But, there's actually a cell phone that you can get for elderly family members, that only allows them to call you, only accepts calls from you and you can leave voicemails that if they call outside of normal hours, it'll be your recorded voice saying, hey, you're fine, you're at home. Go back to sleep. I was like, what if we could have an AI with a more familiarsounding voice, not so scarysounding, answer the phone and talk with the person and respond appropriately? Right. Now I think we're a little far off about that. I want to be cautious and recognize AI, in general, is not perfect. It's built by humans. It has human biases, et cetera, et cetera. When he calls my mom to say, where am I, what's going on, are you okay? The AI could say, Irena's upstairs, it's nighttime. You're safe. Can you get back to sleep? If it escalates, it could decide, do I need to call Irene and wake her up because he's not okay or, you know, are there other strategies that have worked in the past? You could even then inform the AI about, like, things that have happened that day. You know, you could say, hey, you went to a doctor's appointment today. He might be a little bit more afraid or he might forget he's got a bandage on his face and she shouldn't touch it. Remind him he's okay, you also have a bandage on your face. It's okay.
So, what is an agent? It's the same as we use the word, "appreciate you sharing." Right. What is an appreciate you sharing? It's software. Right. An agent is basically software that is leveraging AI in some way, to perform actions on� that are relevant to what task your users are trying to accomplish, is how I would describe it.

JASON LENGSTORF: And so to try to contextualize that, when you say, it's like an appreciate you sharing, if I build, you know, I'm working on my website and I, like, install a ChatGPT, like, live chat thing, I'm not really� that's not really an agent, right. That's sort of a� I've added an API key to my appreciate you sharing and it's sort of doing whatever it does. Where does it change from I've created a direct line for my users and my LLM of choice and I'm turning this from just an OpenAI key to an agent? What are we, as developers, adding to this, that sort of causes the transformation?

SARAH GUTHAIS: So, I want to say, Jason, I think you are a better person than a lot of people because I would argue, there are a lot of people out there right now, saying that it's an AI agent, but really, it's just an OpenAI key. Right. I�

JASON LENGSTORF: That, I think, is why this has been such a hard thing to disambiguate because if what you're doing is you've got an OpenAI key and it's a system prompt that says, you're a helpful assistant, that doesn't feel like an appreciate you sharing to me. That doesn't feel like you've created something. That just sort of feels like you wanted to add an LLM into your site. So, this is sort of where, for me, I want to have� and I know that this is going to be a moving boundary, right, because the definitions are very human definitions and everybody's going to say that whatever their thing is, is.
If I copypaste a bootstrap template, I'm using a bootstrap template. I didn't build that website, I sort of used that template to get a website on the internet and there is a threshold where you've crossed where you've customized enough or changed or done some amount of work to it that makes it yours and now you're building a website. And where that threshold is, is subject to a ton of debate because I believe if you write any code, you're a coder and I think that that is a very true thing. But I do think there's a difference between, yeah, I wrote a couple of lines of code that changes from blue to green and I'm a professional coder that's seller code.
You go from "I told the ChatGPT to talk like a pirate," to, "add additional things to this that it can't be with just an OpenAI key."

SARAH GUTHAIS: And I just tricked you into giving the definition. [Laughter]. But, I think that's exactly it. Right. And that's� that's why, you know, I� I have the privilege, right now, to be choosing where my next role's going to be and a lot of people who are in that same place don't want to go into AI because of a lot of reasons. Right. I won't get into all of them, but I do� and the reason I want to is exactly what you just described, Jason, which is...I want our industry� the people in our industry to recognize that these OpenAI keys, or whatever one you're using, are just another tool. It does not replace the thought, the planning, the, um, biases, the constraints of what you're trying to actually build and achieve and influence in terms of people.
For example, the first version� and the one that you'll see today� is not� I would not consider what we're going to build today an AI agent necessarily yet. What I described with the dementia case, I could hook up an AI key and have it answer and say, hi, what's your name? Okay, it's nighttime, go back to sleep. But, what I described was more than that. What I described was that the caregiver, the primary caregiver could input additional information and track the number of times the dementia person is waking up in the middle of the night. It could be leveraged by the caregiver at the doctor's office. It could track certain keywords that say, pain. They're waking up with a headache. So things like that. That contextualized information is the piece that I think makes it an agent. It's that "can it help humans achieve something beyond� with the help of AI?" Just like we do with the help of mobile devices. With any of these things. That, to me, is the difference between just plugging in AI and actually creating an agent.

JASON LENGSTORF: Yeah. Okay. That makes sense to me. And I think I'm� I think I'm following with you and so let's talk through what we want to get done today. So�

SARAH GUTHAIS: Yes.

JASON LENGSTORF: You mentioned a couple tools. You mentioned LiveKit, you mentioned Twilio. Originally, we were talking about it being an agent but we're saying now, today, we're not quite getting to an agent. Maybe let's start with just a high level on what is LiveKit?

SARAH GUTHAIS: Yes. So, I discovered this� I mean, it's a platform. It's open source, first of all, which I love. I'm a big open source fan. And, originally, it started as a platform dedicated towards building realtime audio/video and, like, data transfer experiences. So, instead of needing to always rely on, you know, StreamYard or YouTube or Twitch, you could host this type of experience on your own. Very easily. It's actually incredible how simple it can be. And, they have some, like, quick starts on that.
But, obviously, with the AI boom, like, being able to support, um, kind of then audio/video, you know, audio visual interactions with an AI is where they kind of moved into.
It allows you to have, like, this full control over your infrastructure so that you can make, like, apps that do video calls, livestreaming, multiplayer connections, they have things with gaming, et cetera. So what I liked about it is...it wasn't just spawned out of AI. It was more spawned out of, this part of developing applications, meaning, you know, livestreaming, multiperson livestreaming, et cetera, is fairly locked down still and how could we open that up so that more people can build more interesting experiences with relative ease and the fact that it's open source on top of it is just amazing.
And, towards the end of last year, LiveKit and OpenAI did partnerships with basically LiveKit is, like, the same technology that powers ChatGPT's voicetovoice.

JASON LENGSTORF: Got it. Okay. Cool.

SARAH GUTHAIS: It's a fairly, like� it's open source. There's a lot of free stuff to start with, but it is really robust and used just by massive, massive, massive companies. So...

JASON LENGSTORF: Great.

SARAH GUTHAIS: Yeah.

SARAH GUTHAIS: And then the other tool that we talked about using is Twilio? I think Twilio is betterknown. Let's make talk through what it does, as well.

SARAH GUTHAIS: Yeah. So, basically Twilio is like a software layer in between making phone calls or texts. Whether it's that� you can think about it whenever you call any company these days, you're going to be talking to a robot. That is the example of what Twilio would do. You would make a call. Instead of going to a phone system, where someone is literally recording things, you send it to an appreciate you sharing that receives that call, can understand, can transfer you, et cetera, et cetera, give you additional information.

JASON LENGSTORF: And they can handle text messages, they can handle the "press 1 to go to sales; press 2 to go to whatever."

SARAH GUTHAIS: Yes.

JASON LENGSTORF: They do far more than what we're talking about. They're one of those tools that sort of become 50 things that Twilio can do now.

SARAH GUTHAIS: Exactly. Exactly. They can email on behalf. If you call, it can email�

JASON LENGSTORF: That's right, because they bought Mailgun? They bought one of them. Transactional email. They have an Events platform and lowlevel infrastructure stuff.

SARAH GUTHAIS: And because it's software, you can then kind of do anything, right? Once the phone call or text message or whatever comes in, then you can go off and do whatever you want. So that's how I talk about it.

JASON LENGSTORF: SendGrid.

SARAH GUTHAIS: SendGrid.

JASON LENGSTORF: So we have a high level of what LiveKit is. We've got a high level of what Twilio. Let's call our shot on what we want to build today. What did you have in mind?

SARAH GUTHAIS: Today, what I would like us to do is we're going to build the very simple, quote/unquote, agent and it is just going to essentially be calling an OpenAI, you know  making an OpenAI call and we're just going to have it be fairly simple, which is, like, be kind. Exactly what you said. We're basically just pulling that in. What I think is interesting is not what AI can to do, but how we can start to piece together all the different pieces of technology to create something.
And so, this took me quite a while to figure out how� like, it seems very simple, conceptually, which is, you make a� like, you set up a Twilio account, you make a phone call. Twilio receives it. Twilio knows what software to call. And that software is a Python file that will accept this incoming call from Twilio and then respond with� basically, create this voice AI to then be on the other side of the call, to join the call.

JASON LENGSTORF: So if I'm understanding the architecture today, what we're doing, we're going to create a phone number in Twilio. When somebody calls that number, they'll be connected to the Twilio route, that will drop their call into LiveKit? OpenAI is going to respond back to LiveKit and then back to Twilio.

SARAH GUTHAIS: The phone call will join a LiveKit room.

JASON LENGSTORF: Oh, okay.

SARAH GUTHAIS: Remember, LiveKit was originally an audio and voice platform, so it joins a room. That triggers an agent, that is powered by OpenAI, to also join the room, to which they can have a conversation in that room. "They," meaning the person joined by phone and AI.

JASON LENGSTORF: Twilio is a traffic controller and LiveKit is where the stuff is happening.
We've got about 70 minutes to do this, so I'm very excited to dig in. Let's swap over here. First and foremost, let me direct everybody to Sarah's Bluesky, you can go find a whole lot of information there.

SARAH GUTHAIS: I have a full tutorial coming out. My Markdown is not properly converting to HTML for the headers, so it is done, it is written. But it will be live soon and I'll skeet about it or whatever the Bluesky is.

JASON LENGSTORF: I love that that's sticking. It's LiveKit.io. We're going to be using Twilio, which is Twilio.com. We are going to be working with OpenAI as our LLM today. So that is� that's our suite of tools and then before we get started, let me do just a quick shoutout. This episode, like every episode, is being livecaptioned. If you head over to the Learn With Jason page on CodeTV, you can see the past episodes and all of the live captioning for that. If you want to watch the� if you want to keep the live captions today, you can go to lwj.dev/captions and that will take you right to our captions that are happening now, with Vanessa, who's here from White Coat Captioning. Thank you so much, Vanessa. And these captions are made possible directly through your support, to you who are watching. If you want to help make sure that we can help keep these going and keep CodeTV alive, you can be a supporter. Somewhere, starting at $5, you can help us create more content and keep the live captioners captioning. Thank you for all you do and making this show possible.
What's the first thing we should do?

SARAH GUTHAIS: Click on "try LiveKit" right there, in the topright. And then continue with whatever your preference is...

JASON LENGSTORF: Okay. Let's get in here. And...is that the Terms of Service. And appreciate you sharing name. So, this is going to be� do you a name for this, or?

SARAH GUTHAIS: I'm going to give you a little context in a minute, but you can call it, like, "partner."

JASON LENGSTORF: Okay, I'm going to tag it with LWJ so I know it was tagged on a show. It's not going to let me skip this, is it?

SARAH GUTHAIS: I think it does.

JASON LENGSTORF: Okay, good.

SARAH GUTHAIS: A lot of reason� just a quick context, when they have those surveys� I know you know, Jason. In particular, when it's open source projects, I like to fill them out because it helps those open source projects where they can continue to support the community so if you're doing this at home, and not on a livestream, I encourage you to do that.

JASON LENGSTORF: Yes, it is really useful unless it is a large company, you are guaranteeing the sales team is going to reach out to you. [Laughter].

SARAH GUTHAIS: As you can see, with just a couple clicks, you are already in LiveKit. They have some, like, guides. They have some playgrounds. You can even click on the OpenAI speechtospeech playground, the third one, yeah. You'll need your key, so you might want to do that.

JASON LENGSTORF: Let me turn off the screen and put that in I am connected. Let me bring up the screen again. Okay. So, now it says "connect."

SARAH GUTHAIS: So when you do "connect," this is now a room, basically, and those are the instructions that it gave to OpenAI. And, you're connected voicetovoice, right. It's listening to you. It can respond to you. It's got a transcript on the side and this kind of just gives you all of the visuals of what you can do within the LiveKit API. So, I really like this as a really simple playground to start to understand.

JASON LENGSTORF: I'm trying to quickly put together all of the pieces here, so that I can broadcast this through.

SARAH GUTHAIS: Yeah, yeah. And if you want, I can get some of the stuff on the screen. On the configuration panel, it's telling you which AI model it's using. It'll give you, kind of transcripts and you can decide, like, which� I guess� "model" is the right word for the transcript. You can choose which voice you use. It'll respond in audio, text. You can change the temperature, how warm, how friendly, how straightforward and how logical it is. And then the turn detection is something that I really love. Turn detection, with AI� with voice AI. You know when you're talking with ChatGPT and you're thinking and it starts to respond and it starts to respond, their turn detection is a little bit smarter than that. It can understand where you are in a sense. And then it'll continue on there.

JASON LENGSTORF: So I just switched to where� are you getting doubledaudio from me?

SARAH GUTHAIS: Yes.

JASON LENGSTORF: Okay. Let me see if I can fix this...so, this is� how do I make this not...hmmmm...what if I put you into here...and turn these off? Nothing but excitement. Let's see, how do I get� nope, that's not it. Uh, let's see...how do I...you can still hear me right now?

SARAH GUTHAIS: Yes.

JASON LENGSTORF: It's not doubled?

SARAH GUTHAIS: Yes, correct.

JASON LENGSTORF: I need to figure out how to get this� what about this. Can you hear me now?

SARAH GUTHAIS: Yes, yes. Still double.

JASON LENGSTORF: But I can't hear you. Great. Dammit. How do I...if I get you...you go away...and then...you go here...this becomes the monitor...I want you gone...you go here...now, do I hear you?

SARAH GUTHAIS: Testing, testing, testing.

JASON LENGSTORF: Still don't hear you. Dammit. How do I make this work? I didn't really think this through. I should have had� let me see if I can get this�

SARAH GUTHAIS: Testing, testing, testing, testing.

JASON LENGSTORF: Can I hear you now?

SARAH GUTHAIS: Testing, hello.

JASON LENGSTORF: And you're getting doubled audio again?

SARAH GUTHAIS: Correct. Correct.

JASON LENGSTORF: Let's see...what if I do... this? Okay. We might be stuck where I don't have a way to�

SARAH GUTHAIS: Share the audio back.

JASON LENGSTORF: To share the audio. Because�

SARAH GUTHAIS: I think that's fine because when I� I'll show a demo, right now, if you want. I'm not going to share this number, just because I don't want the whole world to start calling it and then I get charged a ton. [Laughter].

JASON LENGSTORF: For sure. For sure.

SARAH GUTHAIS: But I'll put this on speaker real quick and one question was, does it support Spanish? Yes. So, it� you can put in different language codes. Especially since the backend, like, you're using the ChatGPT model. Right. So, yes, it'll support basically whatever ChatGPT supports.
So, this is the one that I built. The reason I call it "partner," so I live alone in the middle of the woods. My daughter lives with me but for almost the entire summer, she's with her dad. I want to watch Yellowjacket, I don't want my daughter to watch it. She's 7. It would be nice to talk to another adult about it or watch it with them or randomly, it's supersunny out today. I was, like, beyond the dementia case� because that has a lot of implications if the AI says something wrong. What could I build that's less risky? I built this called Partner. The idea is, like, you know, when you're driving, you don't want to be connected to an appreciate you sharing, but you could have a phone call going. Or if you're on a walk, again, a phone call somewhere is easier. You've got, like, headphones in and you can kind of chat with someone, but not every time� your friends aren't necessarily available at the same time, so what if you could have something you could call, that would remember who you are, remember the context from before and also chat with you about stuff. Right.
So, this is my very simple one. [Phone ringing].

SARAH GUTHAIS: Hi, there, what's your name? I'd love to have a chat with you.

SARAH GUTHAIS: Hey, it's Sarah. How's it going?

SARAH GUTHAIS: Hi, Sarah. I'm doing well. Thank you. How about you? How's your day going?

SARAH GUTHAIS: Obviously, it's very simple. You're chatting to an AI. That was just calling a phone number. That's not an appreciate you sharing. I'm not connected to the internet� my phone is connected to the internet, but I'm using an appreciate you sharing. It just answers and it's just easy, so that's what we're going to build today. You can try out this playground if you want, but why don't we get started on actually trying it out.

JASON LENGSTORF: Works for me.

SARAH GUTHAIS: You've got your LiveKit cloud, this is what this view is. We're going to create a project, so, um...you should be able to go to� thank you. Let me think...I think it's under "sandbox." Yes. I think. Overview...

JASON LENGSTORF: I can create a sandbox appreciate you sharing, it looks like.

SARAH GUTHAIS: I'm trying to remember now. This is the one part that I'm like, wait, how did I create this initial project? This one is their new one, where you can call directly. Sorry, that's what it is. This is where I always forget. It's at the very bottom. I always forget, by the way, that that's where their project list is. For some reason, that doesn't make sense for me. We already created the project, that's why I'm confused. We've in the project now. You might want to do this offscreen. You're going to go to Settings and we're going to go to Keys and we need to create a key and�

JASON LENGSTORF: One doesn't exist. I'll go here�

SARAH GUTHAIS: I guess it did exist, sorry.

JASON LENGSTORF: I'll roll this key real quick and we'll revoke.

SARAH GUTHAIS: You're going to need your project name, your API key, your secret, et cetera.

JASON LENGSTORF: There was a key onscreen, but not a secret visible.

SARAH GUTHAIS: Correct.

JASON LENGSTORF: Taking a couple quick notes offscreen about which one is which. This is the key...

SARAH GUTHAIS: Yes. The dogs are here. Oops, that's the wrong� come here, JackJack. This is Jack. This one's Jack and, Molly, come. Molly's under the desk, so she's coming around. Come here, mama, come here, mama. Okay, okay, that's enough. Thank you. There's a Ruby somewhere. Come here, Ruby. I had locked the big pups out and they snuck in. There's three of them. I won't show you the three cats and the five chickens and the two [Indiscernible] and the snake right now.

SARAH GUTHAIS: The two [Indiscernible] oh, god dammit! Hold on while I revoke all of those and try again. Uhhh. I was so careful. How happy is Vinny right now? [Laughter].

SARAH GUTHAIS: They're just� oh, my gosh. Hi. Hi. Hi.

JASON LENGSTORF: Okay. There's my environment variables.

SARAH GUTHAIS: We just got a frog, too. But my daughter just caught it randomly and I'm like, I don't know how to take care of frogs, dude. So, we're trying that right now, as well.

JASON LENGSTORF: Okay. Now, I have a key and it's not still onscreen.

SARAH GUTHAIS: Perfect. [Laughter]. Perfect. Okay. So we're just going to do some of the basic stuff and let's just� so, we also have a LiveKit project URL, so if you go back to your settings and just under Projects�

JASON LENGSTORF: I've got that.

SARAH GUTHAIS: There's an SIP URI.

JASON LENGSTORF: What is SIP? It says a little thing. Oh, where the docs are.

SARAH GUTHAIS: Session Initiation Protocol. Basically the thing that will create the room. It's the thing that creates a session and allows when a room� when you connect to a room, it can help, like, initiate other things. It is often used with, like, the phone call integrations.

JASON LENGSTORF: Got it. Project URL and the SIP� to people say "SIP"?

SARAH GUTHAIS: I don't know, I say "SIP."

JASON LENGSTORF: I've got my keys offscreen. I've got my OpenAI key ready to go. Do we want to start here or do we need Twilio stuff first?

SARAH GUTHAIS: Let's do Twilio stuff first. Just because then, we'll have everything we need to do anything. So we're going to head over to Twilio. Go to console.twilio.com, and create a new project...

JASON LENGSTORF: Here we go, Twilio home...

SARAH GUTHAIS: And then� at the top� or, yeah, Account Dashboard. It should be� ummmm�

JASON LENGSTORF: API keys.

SARAH GUTHAIS: My screenshot seems old, hold on. They, like, just changed it. [Laughter]. No, mine still shows it. Okay. Up at the top where it says "your account," Jason@Lengstorf� that's weird. I guess the project is an account, is what it is.

JASON LENGSTORF: I'm in a very old system, so I should�

SARAH GUTHAIS: Yeah, create a new account. That's� in my head, that's the project because it's basically� think about it like an account is going to be, um� if you think about one of the most common ways people use Twilio is, like, a call center, for example. You want to have, like, one account for that call center business. Right. And so, that's� it is kind of like a project, in that sense, I don't know why they use the word "account." If there's multiple users using this, does that mean that multiple rooms get created? Yes. That's what the SIP does. Every time it gets an incoming connection request from Twilio, it creates a new room, so multiple people can call at the same time. My thought, for making this an actual agent, is that you� when the room� when the call gets connected to the room, we know what the incoming phone number is, not from Twilio, like, not the Twilio number that you called, but the actual person's phone number, so then you could create just a very simple database that says, you know, the is what I know about this phone number and then you could say, every time someone calls from this number, instead of asking for your name every time, it would go, hey, Sarah, how's it going? Is it sunny today?

JASON LENGSTORF: Quick question. Are we hosting on Twilio or not?

SARAH GUTHAIS: No.

JASON LENGSTORF: I'm in. Got a new project. We got our $15 trial bucks.

SARAH GUTHAIS: So get the phone number.

JASON LENGSTORF: Are they going to make me do the whole "verify you exist" thing?

SARAH GUTHAIS: Make sure you delete this afterwards so that you don't get charged for things. So, now you've got a phone number and we need your� to keep track of your phone number, your account SID and your Twilio offtoken. Those three things you need. I think under Phone Numbers, Manage and Active...there, we should be able to find all that information. We'll do that stuff in a minute, but�

JASON LENGSTORF: I know it's right here. Properties�

SARAH GUTHAIS: There's your phone number SID and your Twilio offtoken.

JASON LENGSTORF: Which I can grab in a second.

SARAH GUTHAIS: Yeah.

JASON LENGSTORF: Twilio SID is going to be this one that I just grabbed. Whoops. Also, I'm just going to grab the number so that I have it.

SARAH GUTHAIS: Perfect.

JASON LENGSTORF: And then, we need to get, um�

SARAH GUTHAIS: Make sure you take it offscreen.

JASON LENGSTORF: Yeah, take it offscreen.

SARAH GUTHAIS: I think it's going to be over on the topright side, because it's the full Twilio one.

JASON LENGSTORF: Or is it admin?

SARAH GUTHAIS: I think�

JASON LENGSTORF: User Settings?

SARAH GUTHAIS: Organization, account...

JASON LENGSTORF: That's not right. Twilio home. I know you're here. I've seen you before.

SARAH GUTHAIS: And you know what? We need the account SID so if you go to Admin, which is admin.twilio.com�

JASON LENGSTORF: There we go. Here's our account SID.

SARAH GUTHAIS: Yes, and then your token.

JASON LENGSTORF: We don't need the phone number SID?

SARAH GUTHAIS: Correct.

JASON LENGSTORF: Account ID is going to replace this one. I can copy this without showing it.

SARAH GUTHAIS: Yes.

JASON LENGSTORF: So, Twilio offtoken. Thank you, Twilio, for making that invisible by default, because there are Vinny's around. And then Twilio phone number, have already.

SARAH GUTHAIS: Yes. And those are the three things we need from there.

JASON LENGSTORF: I'm going to copy that format because I have a hunch that use it that way. I have an OpenAI key, I have my LiveKit URL and secret from LiveKit and I have the SIP. From Twilio, account SID, token and my number.

SARAH GUTHAIS: Perfect. The question about the scaling, this is somewhat dependent. So, I think Twilio does have help with some. Like, just the ability to have multiple people make the calls, but the scaling is also how many rooms are being created, so that's happening on LiveKit, as well.

JASON LENGSTORF: It's kind of a bound thing. Probably, Twilio's not going to be your bottleneck.

SARAH GUTHAIS: Yes. It's going to likely be LiveKit and what you want to kind of set up there.
All right. So, next, what we're going to do is we're going to create� and I don't know how other people pronounce this, a TwiML bin. It's a lightweight serverless�

JASON LENGSTORF: Is somebody hitting the trial endpoint, please stop so that this demo works. [Laughter]. Sorry, where did you want me to go?

SARAH GUTHAIS: Yes, so you can just search in the top bar, where it says "jump to," TwiML bins."

JASON LENGSTORF: Bin.

SARAH GUTHAIS: Create TwiML bin. So we're going to create one. You'll get it a name and then the XML code is very simple. I'm going to send it to you in a private chat.

JASON LENGSTORF: Oh, I see. Response, and say...I'm going into the private chat to grab this code.

SARAH GUTHAIS: And you'll replace your SIP with� your phone number and SIP there. All it's going to do is when this bin is hit, a response will happen, where it will dial to this session initiation protocol, sending in a username and password. That is something that you set, so whatever you want. This was something� I will say, maybe I'm dumb� and that's okay� it took me forever to figure out what this username and password should be until I realized it is whatever you want it to be. Your SIP should be your phone number and your SIP URI.

JASON LENGSTORF: It's already got an SIP here. Do I drop it here?

SARAH GUTHAIS: Yes.

JASON LENGSTORF: Username and password is whatever we want. We want this to be something hard to guess so in the future, people aren't making a big mess out of our� out of hitting this directly. For now, we is say, "test and test."

SARAH GUTHAIS: Yes. So this is going to be where, like, you don't want people to join� be able to join the room unless they're coming from this Twilio or create a room, rather. You don't have to have a username and password. If somebody gets your SIP URI, they can join whenever they want.

JASON LENGSTORF: Got it. Okay.

SARAH GUTHAIS: Then click "save." And we should be good. And you can see, just real quick, it says "valid voice TwiML," the bottom of your code block. You always want to make sure it says that. If it doesn't, check to see what you've done.
Now we are going to make sure that our phone number is actually attached to this bin. This is a serverless entrypoint. So go back to Phone Numbers, Manage and Active and click on the number. And then this is where we're going to set this configure. Set to a bin. Set the bin.

JASON LENGSTORF: One thing I do have to give Twilio props for is they're very good at making this part intuitive. I have not ever used a TwiML bin before�

SARAH GUTHAIS: Oh, I like that.

JASON LENGSTORF: I was able to intuit that that was going to work.

SARAH GUTHAIS: Perfect. Then we're going to just go to the Configure tab.

JASON LENGSTORF: We're not using the messaging.

SARAH GUTHAIS: No.

JASON LENGSTORF: So now I'm going to Configure.

SARAH GUTHAIS: Which I think we were on. Routing is active. That you've configured with that one and it's connected to your bin.
Now if you go back to Active Numbers...

JASON LENGSTORF: Just All Active Numbers.

SARAH GUTHAIS: See under Active Configuration, it shows it's connected. So that's all you really need to do to make it connected.

JASON LENGSTORF: This note, I just saw, this makes it look like it's not going to work. We can call the number, but it can't go out. This is an antispam verification. It will connect to LiveKit but it won't do anything because we haven't set anything up on LiveKit.

SARAH GUTHAIS: Correct. You can try it. I don't know exactly what will happen. I think you just join a room that doesn't do anything or it might just ring. It won't talk to you yet, because we haven't told LiveKit about OpenAI yet.

SARAH GUTHAIS: You have a trial account. You can remove this message at any time by upgrading to a full account. Press any key to execute your code.

SARAH GUTHAIS: That's just Twilio.

JASON LENGSTORF: Oh, that's Twilio. That's Twilio, letting me know that I'm definitely not paying for this. [Laughter].

SARAH GUTHAIS: Exactly. One thing real quick, if you go back to your TwiML bin, which you might� you might have to go�

JASON LENGSTORF: That's not it. I have to search for it again.

SARAH GUTHAIS: Once you search for it again, don't click on Create it this time. You can add.

JASON LENGSTORF: Oh, yeah, you can pin this.

SARAH GUTHAIS: Exactly. Before and after the dial, if, for some reason, something's not working, one debugging tip, just for you all, is that you can put in a Say command, just like it shows kind of on the side. Say, I would say "connecting you to an agent now." Or whatever. This is going to be on the Twilio side, but when I was debugging, this was superuseful for me to be able to just know whether it was Twilio that was having a problem or it was LiveKit or something else. Yeah.

JASON LENGSTORF: Get you that really� that really helpful� like when you get through an automated system and you think you're getting somewhere and it's like, goodbye, and it hangs up on you. [Laughter].

SARAH GUTHAIS: Exactly. If we do this right� "right" is dependent upon how you want your application. You could hang up before that "goodbye" would ever happen.

JASON LENGSTORF: We know if we get here, we technically broke something.

SARAH GUTHAIS: Exactly. You can take those out, if and whenever you want.

JASON LENGSTORF: Once we feel like it's confident it's working� this is our console log, basically?

SARAH GUTHAIS: Exactly. Exactly. Are we ready to write some code?

JASON LENGSTORF: I'm so ready.

SARAH GUTHAIS: So, go ahead and open up whatever code, you know, editor you want to use and the first thing that we're going to do is, offscreen, you're going to create� we'll create a folder.

JASON LENGSTORF: Yeah, let's set up a project. I'm going to go into GitHub, Learn With Jason. Do you want me to use a specific framework or anything for this?

SARAH GUTHAIS: It's going to be Python, but it's literally like one file.

JASON LENGSTORF: How do you check the version of Python, because this could be very funny.

SARAH GUTHAIS: Good question. I think if you just type in "Python 3."

JASON LENGSTORF: Python 3...

SARAH GUTHAIS: And, yeah, "v."

JASON LENGSTORF: What did that just do? Oh, it did so much. You� oh, god, help.

SARAH GUTHAIS: Oh, so you are in. So, yeah, you can just get out now. Yeah. You do have Python.

JASON LENGSTORF: Wooo. Good. All right. [Laughter]. I have Python, so I'm going to make a new directory. [Laughter].

SARAH GUTHAIS: Yes. [Laughter].

JASON LENGSTORF: We'll go, LiveKit, Twilio...

SARAH GUTHAIS: And you're going to create, um...a�.env and I'll send you each� basically, you just� the easy way to do this is� I've put these� you can copy and paste these from our private chat, because it doesn't have any codes in there yet, and then you can go offscreen and replace all of those.

JASON LENGSTORF: So, here is...oh, okay. All right. All right. All right. So, we've got this one� oh, no, don't be like that! Gotta get through all of these and then one more for the LiveKit stuff...

SARAH GUTHAIS: Sorry.

JASON LENGSTORF: No, it's all� this is a StreamYard shortcoming where they don't allow any line breaks.

SARAH GUTHAIS: So while you're copying and pasting those, basically what is in this ENV file is you're Twilio account, SID, your Twilio phone number. Then you have your TwiML username and passwords, then we have our LiveKit URL, that's the one that starts with "wss." We have our LiveKit SIP URI, and then our LiveKit API key and our LiveKit secret. So...three, kind of secrety things from Twilio, the username and password and then four things from LiveKit.

JASON LENGSTORF: Got it. And I'm just slowly copypasting all of these things out. What I don't have in here is an OpenAI key. Do I need to worry about that?

SARAH GUTHAIS: Um...you do not because we're going to set that in...no, because you� when you created the project, in LiveKit, you put in your OpenAI key then. So, LiveKit�

JASON LENGSTORF: It's already on LiveKit?

SARAH GUTHAIS: Correct. That's what's nice about using the LiveKit cloud, in my opinion, it's handling a lot of that configuration. LiveKit is open source so you can do this however you want.

JASON LENGSTORF: Got it. My�.env is in here. If I make a new file, that file should be called what?

SARAH GUTHAIS: Let's make requirements.txt. And we just have three requirements. I put them in the chat, but it's going to be LiveKitAPI and dot env and then Twilio.

JASON LENGSTORF: Is this the Python equivalent of a package JSON?

SARAH GUTHAIS: Yes just save that and we'll do that in a minute. So now we're going to create a new file called "inbound_.

JASON LENGSTORF: This is sneaky, I didn't realize I was going to be learning some Python.

SARAH GUTHAIS: The agent, itself, is a backend piece of software. You're not necessarily� you're deploying access to a agent, right. And so anything that you write around it, yeah, you can, like, one idea that I had is that you could actually talk to an agent on my website and be like, tell me about Sarah. And I'm actually building that, right now, as well, once I get the blog to render properly.
Anyways, okay, so, this is our inbound trunk. So what this means is when Twilio is calling us� sorry. When you call the phone number, and the TwiML bin connects to LiveKit and, thank you, Vinny, Twilio Markup Language. When it connects to LiveKit, it's going to make a call to this Main function.

JASON LENGSTORF: Got it. Great.

SARAH GUTHAIS: The first thing that you're going to need is some imports so we're going to import Async.io. This is just input/output, being able to do it async. Import OS to get to environment variables. From LiveKit, import API. This is just the API. And then from dot env, the word, import load_env. Dot, dot.

JASON LENGSTORF: Oh, oh, oh, oh, right.

SARAH GUTHAIS: Yep. This is just being able to� where we use� this is just actually loading the environment variables into our running space, basically.
Okay. So those�

JASON LENGSTORF: This was a big miss by JavaScript. Like, why? Why didn't we do this? This makes so much more sense.

SARAH GUTHAIS: Yeah. It's� it's� yeah�

JASON LENGSTORF: This could have been us.

SARAH GUTHAIS: I know. I'm more of a backend person, so frontend stuff confuses me a lot, to be honest, like, choices made. [Laughter]. This whole entire file, this is going to be one function and it's fairly simple. It's an async function, asyncdef and (main]. We don't have types here, which is why we are saying "define." We are defining Main, just like we would define any other variables. So the first thing that you're going to do is call load.env. And then, no semicolons and make sure you got your tabs correct. All right.

JASON LENGSTORF: This is going to be a test. This is going to test me. [Laughter].

SARAH GUTHAIS: The next thing we're going to do is create an instance of the LiveKit AI and the LiveKit will automatically read in environment variables. So you're going to create LiveKit_API and that's going to equal API.LIVEKIT.API. And then, now, we're going to create an inbound trunk object and this one is basically going to be the thing that's connecting the TwiML bin to LiveKit. And so, we'll call this, you know, yeah, that's fine.

JASON LENGSTORF: The LiveKit API we just created?

SARAH GUTHAIS: Yes. That's� sorry, yeah, you're right. LiveKit API. Sorry. Yep. And then, I sent, in the private chat, just to get the capitalization correct. SIP Inbound Trunk info. We're saying, this is� you put a T instead of a D on "inbound."

JASON LENGSTORF: Oh. Is this a round boi?

SARAH GUTHAIS: We're going to have four parameters. You can enter, if you want, and it'll automatically tab in for you. Perfect. We're going to have "name," "name=" and whatever you want.

JASON LENGSTORF: Like, with spaces and stuff?

SARAH GUTHAIS: With spaces is fine.

JASON LENGSTORF: Okay.

SARAH GUTHAIS: And then we're going to have an auth_username� I come from a Java, originally, background. Python sometimes even wonks me out.
You've got an "e" in username.

JASON LENGSTORF: Username.

SARAH GUTHAIS: The other one.

JASON LENGSTORF: And then the other one is the password.

SARAH GUTHAIS: And then, the last one that we're doing is, um, enabling Krisp. And�

JASON LENGSTORF: You capitalize your booleans in Python.

SARAH GUTHAIS: Yes, yes. Krisp is just noise cancellation. It helps to make sure you're getting a little less dog noise, as we're getting on this now.
Perfect. Okay. So that is going to create the inbound trunk and notice that we had used the SIP URI in the TwiML bin with the username and password. We are connecting to that same SIP through this project and so, we just make sure we have the username and password in here as well.

JASON LENGSTORF: Got it.

SARAH GUTHAIS: The next thing we're going to do is create a request object with the inbound trunk, so this is basically making sure that we can request that, like, an agent is going to join this� this call, as well. So, request=. It's "create SIP inbound request."

JASON LENGSTORF: This is our LiveKit API and it was "create SIP inbound trunk request."

SARAH GUTHAIS: We have one parameter here: Trunk=, and then your inbound trunk.

JASON LENGSTORF: Got it. That makes sense to me. Look at me, I'm a Python pro. [Laughter].

SARAH GUTHAIS: You're doing great. And the next thing we're going to do is actually create the inbound trunk with LiveKit. So...this is...you can� I had it just kind of almost, like, replacing my inbound trunk. Yeah.

JASON LENGSTORF: Like this?

SARAH GUTHAIS: Yeah. Because basically, first, we just needed the info and then we're creating the request and now we're actually creating the real trunk so maybe, it would have been better, instead of calling it "trunk," called it "inbound trunk info." We're going to do an "await" because we need to make sure this connect. It's going to be your "livekitapi.sip," all lowercase this time. This is one thing about LiveKit� and something that I want to see if I can help them with. Their docs and stuff and API is not always consistent, so,�.sip and dot� this is all lowercase with underscores, okay. Sip_create� sorry, sorry, not sip. I messed up. Create_sip_inbound_trunk, and that's where we pass in the request as a parameter.

JASON LENGSTORF: Got it.

SARAH GUTHAIS: And then that's going to basically send out a request to any of the agents that are connected to your LiveKit, enabling them to join this room.
And then, before you end anything, always make sure you close it so then we're going to do "await" and then LiveKit_API.aclose, no space, and then "clo" and two parenthesis.

JASON LENGSTORF: It's Italian, aclose. [Laughter].

SARAH GUTHAIS: Exactly. All we're going to do is run this. In Python, at the very bottom, not tabbed in, we're going to say, asyncio.run and pass in Main. Okay. So this will ideally actually call in� when you make a call, you'll join, it'll create a room for you and the phone number, from Twilio, will be connected to this room. So the only thing we're missing now is to add in the agent and we're almost done. So, let's save this file and we're going to create "dispatchrule.py." This one, we have the same imports.

JASON LENGSTORF: I can just copy them.

SARAH GUTHAIS: Exactly.

JASON LENGSTORF: Noooo!
[Laughter]. Nobody do anything bad with that for just the rest of this demo.

SARAH GUTHAIS: We're almost done!

JASON LENGSTORF: Are we doing Main again?

SARAH GUTHAIS: Asyncdefmain. And then load_dotenv. The word, "dot" in front of it. And then, you're going to do another LiveKit API, so, whatever you want to call it. LiveKit API�

JASON LENGSTORF: I can probably just copy that out?

SARAH GUTHAIS: Yep, exactly.

JASON LENGSTORF: So close to getting through all of these API keys without blowing my cover.

SARAH GUTHAIS: Next is we're going to create a SIP dispatch rule object and so what this means is it's going to request a private, or individual room for each call, and we're going to prefix that room with the call to indicate that it's a call. Like, if you're using LiveKit for a bunch of other stuff, like apps, we want to make sure this one's for a call. Rule = LiveKit_API.sipdispatchrule. Send in dispatch_rule_individual. And that's going to equal API.� I don't love this API. Yeah, sorry. LiveKitAPI.SIPDispatch. Some pass in one more parameter that says, room _prefix equals and (call dash). We're going to say agent=, and call API.RoomAgentDispatch. And pass in one parameter, agent_name. I called mine Outbound Agent.
Then we're going to configure the room to add the agent to it. So, for that, we're going to� I called it "room_config. LiveKit API.RoomConfiguration and pass in one parameter, Agents. And it's going to be an array, so square bracket.
Next, we're going to create the request with the rule and room configuration. So, our request is going to equal LiveKit.CreateSIPDispatchRuleRequest. We'll pass in rule=rule and roomconfig=roomconfig.

JASON LENGSTORF: So Python doesn't do the shorthand expansion for stuff like this?

SARAH GUTHAIS: No.

JASON LENGSTORF: Okay.

SARAH GUTHAIS: Then we're going to create the actual SIP dispatch rule with LiveKit, like, we're going to register it, basically. Hey, Brian! This is dispatch=await and you're going to do your LiveKit API.sip.create_sip_dispatch_rule, with the parameter, "request." And then we're going to await LiveKit.API.aclose, for when we're done. And then you're going to run the Main function, not tabbed in so it's going to be asyncio.run. And Main with the parenthesis.

JASON LENGSTORF: Okay. We don't have to do anything with this dispatch?

SARAH GUTHAIS: Nope, because you're just awaiting it. It's just going and then you close the whole entire API. So you're good.
All right. Oh, very cool, very cool.

JASON LENGSTORF: Okay, I have an inbound trunk and that is setting up the Twilio trunk, which gets us connected to the SIP and when somebody actually connects, we are setting them up with this room and hooking them up with an agent. Now we need to design the agent� we're not designing the agent?

SARAH GUTHAIS: Right now, we're just going to have it be� hang on, where are we? Oh, wait, what did I do?

JASON LENGSTORF: Do we need to do anything in here to say, hey, when you get a call�

SARAH GUTHAIS: Yes, we do. Make sure you have your dispatch rules. Okay. So, this is where we're going to need our API key. I just realized we are totally running out of time, but we can do this. We can do this. So, we need� let's create, like, a different folder for� in your VS Code. Let's create another folder because this is where our agent's going to actually be.

JASON LENGSTORF: Okay.

SARAH GUTHAIS: And let's create a file.env and the ones that you're going to need are these ones...

JASON LENGSTORF: Okay. Let me just pull this offscreen for a second so that I can do all these bits. The LiveKit API keys and the OpenAI key. I got it.

SARAH GUTHAIS: And, yeah, the ones he's adding in there are the LiveKit URL, the LiveKit API key, the LiveKit secret and the OpenAI key.

JASON LENGSTORF: OpenAI...API key, is what you called that. Okay. LiveKit API, secret API key, URL. Got all those. Saved. Closed. Definitely not going to show these to anybody, being very good with our security today. I've got that.

SARAH GUTHAIS: Yes. Now we're going to create an instructions.txt. And, this is what I gave mine, you can just copy and paste it for now, if you want. Basically, it says, you're helpful. You are talking to an adult who lives alone. Tear. [Laughter]. Calling you to chat about your day, you should reference things, et cetera. Now we're going to create the agent.py file and I think we could do this fast. So, I'm going to send you all of the imports and as you're kind of copy and pasting and newlining, I'll describe them. So, we are going to get, um, annotations� why are we using annotations again? I can't remember exactly we're using that one. I think that one's for reading things in. OS and logging. Logging is so we can log what's happening with the agent. OS is reading. We've got the LiveKit RTC, where is that one? RTC?

JASON LENGSTORF: RTC is for the realtime stuff?

SARAH GUTHAIS: Yeah. There it is. Wait, no, that's not it. We'll find it when we get down there. And then we're bringing in all the agent stuff, so autosubscribing, meaning it's automatically joining into the room, be able to talk to the CLI, connect with an LLM. It's a multimodal agent because we want to do voice and text potentially. We'll bring in the OpenAI plugin and dot env.
Next thing we're going to do is load.env. We don't need that yet. We're going to load the environment variable. So, load.env. Yep. And then, I'm just going to send you some logging code, just to make sure. This is just useful so you initialize the logger with the agent and we're going to open the instructions.txt and read that information in and I like to log that just so we kind of know where we're at.
Now you're going to do something slightly different. It's async def, but Main Entry and provide CTX which is context and job context.

JASON LENGSTORF: Like that?

SARAH GUTHAIS: Job context, capital J and then the actual word, Context, with a capital C. We're saying that the CTX parameter�

JASON LENGSTORF: That's a type.

SARAH GUTHAIS: Exactly. Exactly. Okay. We're going to log a little bit and then also make sure that we have our OpenAI� OpenAI API key and our LiveKit API key. And then we want to actually join a LiveKit room with audio only. And then, we want to, um� don't do anything until we have a participant join. Yep. And, these should all be tabbed in, which is why you're getting the error.

JASON LENGSTORF: We're inside of our function. I understand.

SARAH GUTHAIS: Perfect. We're going to set up our OpenAI realtime model. So that's this stuff, right here. So, AI model is going to be an OpenAI. This is just OpenAI stuff. But we're going to send in the instructions. I chose Shimmer audio and text and sent in my key.

JASON LENGSTORF: Okay.

SARAH GUTHAIS: We're going to create our multimodal assistant with LiveKit. It's going to be an AI model and it's going to start in the context of that room, or in the room for that context. We're going to initialize the session and create, like� initialize and create, like, the session and the conversation. So, the session instance should be your AI model is, like, knowing that it's a new session. The conversation should basically be an LLM chat, it's a user and it's just going to be� begin the interaction consistent with the instructions we sent and call "create." And then, the last thing we need to do� and then we'll actually test this out� is, say "join." So if we are going to call Main, like, if� if Main was called. This is out of the function. If Main is called, meaning that we've done all the dispatch stuff, then we want to actually add the agent to the room, basically.

JASON LENGSTORF: Okay. Like that?

SARAH GUTHAIS: Yes, sir. Looks good. Okay. Are you ready?

JASON LENGSTORF: I'm so ready.

SARAH GUTHAIS: Save your files and let's open up a terminal...and, in your higherlevel one, which is this one here, I think. Where are we? We're not in the agent one yet. We're going to make sure that we're just running the one that's going to create the room first. So, to do that, we need to do, um, Python� hang on, let me grab it for you so that we do it right the first time. Where are you? Where are you? So, Python3mvenvvenv. We create a virtual environment so we're not installing things in your computer, we're installing them on this virtual environment. It should say, virtual environment, blah, blah, blah. Perfect. You see�

JASON LENGSTORF: I see the VN.

SARAH GUTHAIS: Now we're going to do pipinstallrrequirements.txt.

JASON LENGSTORF: Okay.

SARAH GUTHAIS: And then we're going to run our inbound trunk, so we're going to basically make sure that LiveKit knows about this trunk, like, this� these instructions, so we're going to run those...

JASON LENGSTORF: Did I typo? Inbound trunk info.

SARAH GUTHAIS: Uhoh, uhoh, uhoh. Hang on. That should be working. Ahhh. Sip inbound trunk info. That should be right. Hang on. Hang on. LiveKit API. Um, sorry. That actually should be API, not LiveKit API, because we're actually� if you look up at the imports, we're importing API from LiveKit also. That's separate from creating the LiveKit API. It's a little bit of a mess.

JASON LENGSTORF: Got it. Okay. Okay. So, doublechecking.

SARAH GUTHAIS: Yeah, that request should also be API, but then your other ones should be LiveKit.

JASON LENGSTORF: This one should be LiveKit and this one should be LiveKit?

SARAH GUTHAIS: Correct.

JASON LENGSTORF: Okay. Let's try it again...

SARAH GUTHAIS: Cool.

JASON LENGSTORF: No errors assume means it worked.

SARAH GUTHAIS: Correct. We can do the same thing. So the "rule equals" should be "API".

JASON LENGSTORF: That should be API, that should be API. This should be API? This one should be API and this one should be API, but then this one is LiveKit API.

SARAH GUTHAIS: Yes, the last two. The last two� yep. Perfect. Let's go ahead and give it a try.

JASON LENGSTORF: No errors because we're Python geniuses. [Laughter].

SARAH GUTHAIS: Now what we can do is, we can...I think we can do this. Can you C/D into? Let's do a pipinstallrequirements.txt here.

JASON LENGSTORF: I just need to copy the same ones.

SARAH GUTHAIS: I thought we did.

JASON LENGSTORF: We don't have it in the agent. I just have the instructions. Is it just the same one?

SARAH GUTHAIS: Um, it is� yes.

JASON LENGSTORF: Okay.

SARAH GUTHAIS: Cool. And then� which they should have already been there anyways, but that's okay. Now we're going to run Python3agent.py dev. Let's try it. Nooooo! Oh, I didn't get one of the things. Um...this one.

SARAH GUTHAIS: Oh, perfect. Okay. Work, work, work.

JASON LENGSTORF: Import RTC.

SARAH GUTHAIS: Don't tell me they changed it since I did that?

JASON LENGSTORF: We didn't use RTC, either.

SARAH GUTHAIS: Try to remove it? I thought that� I think we might still need it� yeah, we�

JASON LENGSTORF: No model named "LiveKit agents."

SARAH GUTHAIS: Don't tell me� it's still working for me, so don't tell me it's been changed.

JASON LENGSTORF: A brandspankingnew version?

SARAH GUTHAIS: Noooo. Can I see yours real quick? We forget. We're missing one. What are we missing? OpenAI? Let's save that. Let's run pip again and then now let's try, maybe? It might be that.

JASON LENGSTORF: Cannot import RTC from LiveKit. Is it from OpenAI?

SARAH GUTHAIS: It's not, but I was just wondering if that was dependent on an OpenAI one. This doesn't make any sense, could you put a new line in between "import logging" and "from." That does make any sense, but I'm trying to see what is the difference between our files. No. We're totally at time. I don't know why that's not working. That does not make any sense. It 100% should work.

JASON LENGSTORF: You have yours running, that's on this code?

SARAH GUTHAIS: Yes, the exact same code and it's totally working.

JASON LENGSTORF: So probably, then, there's�

SARAH GUTHAIS: Go to "rerequirements" superquick, add in LiveKit agents and regular LiveKit and LiveKit.plugins.OpenAI. Why were these not in my other notes? Okay, now try running pip. Sorry.

JASON LENGSTORF: This is going to work.

SARAH GUTHAIS: It's totally going to work.

JASON LENGSTORF: And the most brilliant photo finish demo we've ever done. [Laughter].

SARAH GUTHAIS: Please work, please work, please work. I'm going to be so sad if this does not work. [Laughter].

SARAH GUTHAIS: No module named LiveKit.agent.multimodal. Did I typo something? Oh, oh, did I typo this? Multimodal? Is it capital M?

SARAH GUTHAIS: No, wait, that one should be right. Import multimodal agent. LiveKit.agents. From LiveKit.agents. Import autosubscribe, job context, LLM. Try� why is LiveKit.agents� did it not actually import?

JASON LENGSTORF: It can't resolve this at all.

SARAH GUTHAIS: Go to the requirements.txt. There it is. Yeah. LiveKit...LiveKit.agents. �.agents, that's why. Yeah, there we go.

JASON LENGSTORF: This is going� this is it.

SARAH GUTHAIS: It's going to work! No, stop it! Did you run pip again? You did, right?

JASON LENGSTORF: I thought I did.

SARAH GUTHAIS: I thought I did.

SARAH GUTHAIS: I thought so, too. That looked like it did more.

SARAH GUTHAIS: Requirement already satisfied. There's no errors.

SARAH GUTHAIS: No, I don't see any errors. LiveKit.agents. LiveKitAPI. LiveKit� regular LiveKit��.plugins.API.

JASON LENGSTORF: Run the pip...

SARAH GUTHAIS: What?

JASON LENGSTORF: Does not like something. Is there, like, an order of operations in these?

SARAH GUTHAIS: Try putting LiveKit� yes. Put LiveKit above LiveKit.agents. Sorry. Duh. Yes, run pip again and then it should work. Yes, of course� what the! What is happening!? Um...LiveKit� oh, but you have LiveKit API above LiveKit, try putting that one below? Oh, my god, I'm going insane. Your multimodal's after agents, yes, it was. Why are you not import multimodal agent? This is so frustrating.

JASON LENGSTORF: Do you want to copypaste your whole requirements file?

SARAH GUTHAIS: Yeah, everything is here, which you're also welcome to share. I think it's private. Yeah, it's private. That's the requirement and this is my imports. I can make this public. Let me make this public real quick, too, that way we can share it with peoples.

JASON LENGSTORF: Okay. So you've got LiveKit.agents...

SARAH GUTHAIS: I'm so sad.

JASON LENGSTORF: Yeah, I'm wondering, is there a way to force refresh or anything? Can I�

SARAH GUTHAIS: What we can do is, let's do this real quick...we're going to� just type in "deactivate." Okay. And then, inside of this one, run, um, Python3mvenv. And then Source. It might have been� so, sourceven/bin/activate and try pipinstall again. Installrrequirements.txt.

JASON LENGSTORF: Stuff's happening now. Wonder if the parent was doing something weird. And then we want Python3agent.py and then dev.
Yes! Okay!

SARAH GUTHAIS: Oh, my god. Call the number! Call the number!

JASON LENGSTORF: I'm calling the number!

SARAH GUTHAIS: Call it, call, it call it.

JASON LENGSTORF: We're going fast, here we go.

SARAH GUTHAIS: Twilio�

JASON LENGSTORF: Did you hear it go busy?

SARAH GUTHAIS: Yeah, that's weird. That would be a Twilio thing.

JASON LENGSTORF: Did somebody burn our credits?

SARAH GUTHAIS: Maybe. It should say, connecting you to an agent.

JASON LENGSTORF: No, we've still got� it says we're busy.

SARAH GUTHAIS: Under Twilio Active Numbers, if you click on your number again, see Call Logs and let's see if� incoming...are we� uhhhh...you're not� so, we have that one.

JASON LENGSTORF: Incoming. That was the first one.

SARAH GUTHAIS: So, why are we not�

JASON LENGSTORF: 8886250086. [Busy signal].

SARAH GUTHAIS: Twilio, what are you doing!

JASON LENGSTORF: Okay, so we screwed something up here.

SARAH GUTHAIS: If you go under Twilio, it should still say, Active, right? Yeah, it still says Active.

JASON LENGSTORF: Says, Active. Going to the thing.

SARAH GUTHAIS: Try your TwiML bin. It should just work. Again, this has nothing to do with, what's it called? LiveKit. Try remove� make sure the Say is happening. Yep. And just say, hello. And make sure you save that at the bottom.

JASON LENGSTORF: Oh, I think I didn't save this last time. We're going to call this, go in speaker mode. It won't even take the call. Let me take this out entirely.

SARAH GUTHAIS: Make sure you save. Twilio, WTF? [Busy signal]. Why is my Twilio number busy?

JASON LENGSTORF: Don't know. Don't know what I did.

SARAH GUTHAIS: This would totally work.

JASON LENGSTORF: Unfortunately, we are way out of time. But, we did hear, earlier, when you called your agent, how it was talking. We know the component parts in the place and we had had prebaked turkey in the oven. If we could figure out why this Twilio number think it's busy. Something weird is now. Right now, it shouldn't even hit. It doesn't even attempt to hit LiveKit. So, this is something� something weird is going on.

SARAH GUTHAIS: Twilio is busyo.

JASON LENGSTORF: This was very informative because seeing how we do get Twilio configured to accept a call that goes into one of these, we can see how this would be really valuable, right, and how we can use this in a number of ways. Like, I� I can think of some funny things that we could do, like making your own version of the throwback movie phone. If I needed to just really quickly get an idea of what my calendar's like, hey, read my calendar.

SARAH GUTHAIS: Yes. So you could just, like, text in the morning and be like, what is my calendar for today. And as you saw, the Python is fairly simple. You could imagine connecting that to any type of services you might create or connect to.

JASON LENGSTORF: Yeah. Throwing some links on the screen here. Make sure you go and check out upcoming episodes of Learn With Jason. I've got Sarah's Bluesky on the screen here so you can go and give her a follow. I assume once you've got that blog post ready, where you walk through this, you'll have a link shared through Bluesky. If you want to dig in, you'll be able to do that. Make sure you go and look at the website for upcoming episodes.
Next week, we're are going to be talking to Ada L�ndh� about how to use SSH for SFTP.
We've had Vanessa hanging out with us, thank you for giving us a few extra minutes. Vanessa's through White Coat Captioning. That is made possible through your support. If you've got it in your wallet to give us, you know, $5, that would mean a lot to us and everybody working on CodeTV. Thank you all for spending a few extra minutes with us. We will see you all next time.

SARAH GUTHAIS: Thank you.