- Hi, I'm Tracy LaTourrette, Colorado's First Lady Fighter Pilot.

Call sign, Jackie'O.

We're here at Wings Over the Rockies Air and Space Museum in Denver, Colorado.

We're getting an inside look at artificial intelligence in aviation.

- It is a moment of wonder, shock and awe altogether.

- We're around the Wright brothers moment.

- We're in a race for military technological superiority.

- How do you methodically explore the unknown.

Question of trust is the heart of the issue.

- At Edwards Air Force Base, they took an F-16 like the one I used to fly, modified it and now it can fly itself.

I'm home, back in the cockpit!

- We can go maneuver relative to another aircraft that doesn't actually exist.

- This is gonna save a lot of pilots' lives.

- We need to seize the day, we have to be really careful with it.

- This is the way that wars are fought and that we need to put real guardrails on.

- It's time to go Behind the Wings.

- Artificial intelligence is rewriting the rules of military aviation.

As planes are programmed to fly without pilots, uncrewed aircraft will define the next era of military and someday civilian flight.

However, human ethical and operational considerations don't always keep pace with technological advancements.

Those working on AI's cutting edge must balance great potential with great responsibility.

- Every time we get a new technology, we have certain expectations and dreams for it.

You may have seen pictures in the late 1800s when we were starting to get the sense that we can make an airplane, we can make a thing with wings that could fly.

And then you got these fantastical pictures of these airplanes shaped like boats, right?

We had paddle wheels, but once the Wright brothers flew and once we actually started doing it, nobody imagined what the airplanes actually ended up looking like.

To me with AI, I kind of feel like we're around the Wright brothers moment.

(music plays, wind blows) - The outskirts of the Mojave Desert in southern California provide an ideal setting for the United States Air Force Test Pilot School.

- The school is a unique institution in that we have a flying operation and a master's degree program.

Our students fly over 35 aircraft in a year.

It's perhaps the most complex flying operation in the Air Force.

- You walked through the same halls that Chuck Yeager walked through back in 1947 when he broke the sound barrier.

- This school is full of history.

It is incredible.

You know, the drive from LA out here to the desert, there's a lot of desert and a lot of opportunity to contemplate.

It immediately strikes you what a perfect place this is to fly.

And then as you drive across the lake bed and you realize all the aircraft that have landed here and have taken off here for the first time, the history of the school is incredible and you can feel it as you walk around here.

But one of the things that I'm very excited about is the future in autonomy and aviation really began with simulation.

Thousands upon thousands of runs with multiple different performers we're conducted in a simulated environment to mature these algorithms.

What makes these algorithms different than a lot of the autonomy you might find on autopilot today on most aircraft is that that kind of autonomy is line by line.

If this then that do this, the kind of autonomy we're talking about, machine learning uses statistical methods to find patterns in human behavior.

It's very powerful and in some ways better than humans, for example, in Chess and Go.

And yet it's not always predictable, it's not always repeatable.

- Artificial Intelligence is not a new subject.

It goes back to the 1950s and 60s to the mainframes.

It predates the internet.

And really what changed around 2010 is Moore's Law finally caught up with us and we were able to get more and more processing.

We were able to do it faster and suddenly raw compute capacity caught up with our dreams of the algorithms.

It is a moment of wonder, shock and awe altogether.

We need to seize the day, we have to be really careful with it.

Would be wonderful if lethal AI weapons never existed.

I am confident that somebody will invent them and if we aren't capable of dealing with them, then we're gonna be in trouble.

- In Washington, D.C. the ethics of military AI are central to the technology's development.

- Your conception of AI and robots as they relate to the military and war fighting might be sci-fi in your head, but it's actually here.

We're becoming increasingly more willing to trust AI, as recently as a few years ago was something that was experimental.

But now as AI is getting more sophisticated and prompting people to think of it as something that could be used in the military, that if the military doesn't develop on its own, it will be outpaced in we have, you know, months, not years before this becomes a reality.

This is something that we should be watching for.

This is the way that wars are fought and that we need to put real guardrails on the idea of empowering weaponry with AI.

That is something that once you create it, you don't take back.

- You worked at the Pentagon back in the 80s.

Autonomy was something that was front of mind.

How have we gotten to this place now where AI, where artificial intelligence is really the name of the game?

- It's been an evolutionary set of technology developments with revolutionary applications.

I was one of the proponents of our uncrewed autonomous aircraft in particular.

And I've watched that technology over the last almost 40 years move to a point where we can increasingly rely on it for some of the functions that traditionally humans have done as operators.

I think it will be a paradigm shift.

I think it's happening as we speak and this is a change that's relatively recent.

- Talk about ethics.

How do we do that on the global stage?

- First of all, we have to design those requirements in.

They're part of the design process and secondly, we have to move fast.

People who are not constrained by that, who don't take the time and the effort to design those kind of constraints onto their systems do have in some ways an advantage.

I think overall we have the high ground here and I think it's the right thing for us to do.

So that's what we are going to do, but it dictates that we move as quickly as possible to integrate these technologies into our systems.

We're in a race right now for military technological superiority.

We've gotta move quickly.

You don't win a race by standing still.

- How do you get to a place where you can trust that aircraft when it's being flown by this artificially intelligent agent and not the pilot that's in the backseat.

- Question of trust is the heart of the issue.

And I think if you look at the trust question, there are several aspects that we have to explore.

First of all, can it do what we're asking it to do in the first place?

Does it do it well?

Can it do it safely and can it do it ethically in a way that conforms to the norms that we as an American military and American people expect?

- Normally you train the AI on the ground and then once you put it in the air, it's no longer learning while it's in the air, but it's collecting data.

Then when it lands again you can use the data it's collected to retrain the AI get better and then have it fly again.

The reason why is that we would like when it's flying that we are fairly confident that we know what it's going to do.

We don't want the algorithm changing mid-flight, at least not yet.

That may be something that comes in the future, but at the moment that's not the cycle we normally use.

A lot of what we learned is that we didn't know what we didn't know.

It's a unknown unknowns.

It looked like it was working well based on all the parameters that we could put on it, but then when they put it in the air, it suddenly will find itself in a position that we weren't prepared for.

- One of the challenges of course in developing simulations is that the reality is very different and so it was necessary to take it from a training environment in a simulation and move it into the real world and that's where the X-62 becomes part of the story.

So we have here the X-62 Alpha, which is also known as the VISTA.

It's a one of a kind national asset.

We looked at this aircraft and realized that it would allow us to take these cutting edge autonomy solutions and to put them into an airborne environment.

- General White part of your job is to figure out how to execute our strategic vision by buying or building the things that we need to move forward in the future.

How do you even get to a place where you're looking at something like the VISTA, a flying simulator?

- So I think Vista is a great example of how we're working to evolve technology over time to meet our needs and it's that first step towards building that trust with our operators and we're verifying that this autonomy is gonna work the way we wanted it to.

When we think about autonomy as an Air Force, we want it to be platform agnostic.

So I should be able to take autonomy off of an F-16 and put it on any platform.

VISTA is our first step on that journey and we're gonna continue to evolve that over time.

- At Edwards Air Force Base, a revolution is taking place in AI piloted aircraft.

At first glance, this F-16 looks a lot like the one I used to fly, but when you look close, there are a whole lot of differences.

Demon, we don't even call this an F-16 anymore, it's now an X-62 VISTA.

So what's new?

- The VISTA stands for the Variable In-Flight Simulator Test Aircraft.

It's designed to be a research test bed for experimental flight controls and autonomy algorithms.

- So inside and out it's a whole lot different than an F-16.

Can we take a closer look?

- Yeah, let's go.

- All right.

So Demon simulators are usually something we use inside.

So what exactly is a flying simulator?

- The X-62A can actually simulate the motions of any aircraft that I can model.

So what that means is a pilot sitting in the front seat here, when they pull back on the control stick, it's going to feel like they're flying a 747 or the Space Shuttle or a T-38 or any number of aircraft that we can program into this simulator.

- So the F-16 is the smallest fighter in our inventory.

How do you simulate larger aircraft like the F-35 or like you mentioned, an airliner size aircraft?

- Well, although the F-16 is one of the smallest aircraft in our inventory, it is also one of the most maneuverable aircraft.

And so anything that isn't as maneuverable as an F-16 I can simulate because I have the ability to make those emotions and those dynamics with this aircraft.

- So you're at test pilot school and you have students only in this case it sounds like the student is the AI agent.

- Whenever we're out testing new autonomy algorithms, we always have two humans on board.

We have a safety pilot who's one of our most skilled instructor pilots and we have a system operator who knows how to operate the autonomy systems on this aircraft.

- Is it true that you actually took out the gun because you needed more room for computers?

- This aircraft doesn't need a weapon because it's not a combat aircraft, it's a research test bed.

We removed the gun and replaced it with all of our research computers so that we have the computing power to run our autonomy and to run our in-flight simulator.

- So I know a lot of the magic happens on the inside, but let's take a quick look at the back.

- So a lot of it's the same as a regular F-16.

The tails are the same size as an F-16, although the software inside the jet can move these in ways that a normal F-16 can't.

We also have special spin chute mounts that were used in the early days of this aircraft when it was a research test vehicle for a multi-access thrust vectoring nozzle.

- So you mean they took the nozzle off of this jet and replaced it with a thrust vectoring nozzle to test for things like the F-22.

- It used to be a research test bed and we did put a thrust vectoring nozzle on this that fed into research that went into the F-22.

But nowadays we use that same research architecture and safety sandbox to test autonomy.

- It's great to see the old F-16 all grown up and I'm dying to see how the cockpit has changed.

It may look a lot like a regular F-16, but the VISTA can actually adapt like a chameleon taking on the characteristics of a variety of a different aircraft, even changing from one type of aircraft to another multiple times in a single day.

This all may sound like science fiction, but here at Edwards Air Force Base it's happening right now.

So Evil, I understand you actually fly this aircraft from the backseat.

How does the AI fit into all that?

- So the AI lives in a computer in the back of the airplane and the front seater is mostly charged with connecting it all up to the jet and then meanwhile, I'm just sitting back there making sure that's all going right and taking over whenever I need to for safety.

- So the front seater then is typically like a flight test engineer.

- The front seater can be a flight test engineer, but later on as the system is working right, we will put a fighter pilot in the front seat who's got all that expertise on that so they could sit there and observe how the airplane is handling, how the AI is doing its job.

So we'll go head up to the cockpit and show you how they changed this airplane to make that possible.

- I'm home back in the cockpit, this is amazing.

So it feels like I'm sitting in my old F-16 cockpit, but this entire aircraft has been reconfigured for research.

- The setup that allows us to do that, that makes it very easy for me to take control of the airplane in the back no matter what makes it easy to kinda replace you with the AI.

My job in the back has hasn't changed.

I'm still there to turn the system off and then fly the F-16 and just keep things safe.

- So there's always a human on board so that you can test the AI in a very safe environment.

- So we're testing these at a very early stage and what we see is fascinating.

We have this problem and that problem is we train these AIs in a simulator and then we take 'em out and put 'em in the real world in this machine, but to watch how the AI fails and then to see how quick they change the training and then change the parameters that the AI works in.

So we're kinda growing the AI in the airplane.

- With the walk around complete it's time to throttle up and go fly!

Fight's on!

We're headed into the flight brief where they'll review the plan for the day.

- So the objective for today is we're going to test Agent Bravo on the X-62A VISTA and we're gonna test it in a number of different basic fighter maneuver configurations.

- All right, great.

So, let's go through the test flow.

- Next up is gonna be test 0.1.

Once I give you that indication, you're going to request the test points.

You're gonna go ahead and start the data recording, you're gonna select the test point and then you're gonna wait that agent to load and after that it's fight's on and the AI is in control.

- Today we're doing a virtual target, so what we're expecting to see is we're set up and we're in a turn like this.

Once everything is ready to go, when we engage, now our airplane with AI controlling is gonna do some sort of defensive maneuver.

While the other agent virtually will be dogfighting with it.

- Just make sure we turn off the simulation before we land so that, you're landing the normal F-16 and then we're ready to go.

- I'll meet crew at the desk in 10 minutes.

I'll see you there and we'll have a good sortie.

- I can't wait to see this thing fly.

(music plays) - Yeah, so welcome to the control room.

This is where we monitor all of our test missions and so we not only make sure that the test pilots up in the jet are evaluating the right systems and they're looking at the right stuff, but we also help keep them safe here as well because we have a ton of data that we can monitor here while they're focused on evaluating the autonomy or the AI.

Test control, you are clear to load test points.

- [Radio] Copy that.

- Over the last year we have been breaking all kinds of barriers in applying autonomous agents in the aerial combat maneuver.

So basic fighter maneuvers.

(airplane whooshing) And that's what we teach at the school is this mindset of how do you methodically explore the unknown.

- Next up is gonna be test 0.3.

We're gonna test the Bravo Agents.

You are clear to load test points.

Right now he's loading the test points and we're just waiting for their indications that they've loaded appropriately.

Test control, you are clear to maneuver.

- We began the aircraft by itself just to see if it would maneuver in a way that we expected and then we put it up against a virtual target.

So we would beam up essentially an F-16 that it would fly against.

And there again we were looking to see did it follow the rules of engagement?

Did it respect the thousand-foot bubble?

Did it behave in a way that we would expect an F-16 pilot to behave?

- Okay, next up is gonna be test 0.4, the Bravo Agent defensive BFM.

Test you are clear to load test points.

So we're actually bringing up a simulated aircraft, which is fantastic because we can go maneuver relative to another aircraft that doesn't actually exist.

So we minimize that safety risk of actually hitting another aircraft because one doesn't exist.

There's the agent making its inputs and turning and it tripped it off.

Test control, point complete.

- So the VISTA's up there flying, you're here in the control room and you can see how the AI is actually acting.

- We can see what the aircraft is doing from here and we can keep track of whether it's doing too abrupt of a maneuver.

Is it too aggressive?

Is it not aggressive enough?

Test control, that is a test point complete.

You're clear to reset the VSS.

- [Radio] Copy that, thanks a lot.

- Only once we were assured that yes, it was behaving according to what we expect.

Yes, it was following all the training rules.

Then we then put it in the same airspace with another aircraft.

(music swells) And even then we started with very slow buildup.

We did bubble passes.

Does it avoid like we expect it to?

Does it avoid the floor like we expect it to and then we put it in an offensive possession.

Does it maneuver to a weapons engagement zone the way that we expect it to?

And then we put it in the defensive and only after all of those and we kept getting the positive signal that yes, it's safe, it's effective, it's ethical.

Did we then put them beak to beak.

We conducted the world's first air to air combat maneuvers in live time, not in simulators between the X-62 and under control by an AI agent and an F-16 that was crewed by a test air crew and then ultimately progressed to what we call high aspect maneuvers where these two aircraft were approaching each other at 1,200 miles an hour at less than 2,000 feet of distance and doing it safely and repeatedly as they engaged on in combat maneuvers.

- Test control, that is test mission complete.

You are clear to RTB.

- [Radio] Copy that.

- So with each flight we're learning about how to update the AI software.

We learned some things today, so you'll go back, make some adjustments, and tomorrow see how the AI performs.

- Some people think that the AI updates itself, but that really isn't the case here.

We're gonna go look at through our data, make some updates and yeah, well hopefully it improves tomorrow.

- You're on the cutting edge of AI.

Tell me a time when this really blew your mind.

- Well, years ago when we first started putting these agents in here, I was flying one of these things and I come back down and we're in debrief and one of my engineers who was in the control room, he says to me, "Hey Evil, these dudes, they were like practically in tears" because these people had spent years and years of their life working on these things.

They thought they were a decade out from having it ever control an airplane and here they were in a control room watching their baby with lots of problems, but still their baby doing things in a real jet fighter.

As an engineer, that's just the best feeling in the world because you get to see how your work is really gonna do and better yet, you're getting the information you need to make it better.

- Where is this all going?

- The future of AI, it's hard to guess.

I think it's gonna do more than we thought it could do.

It's gonna cause headaches, we didn't think we were gonna have.

The power of it is a little bit scary.

It reminds me a little bit of a nuclear power and nuclear weapons in the sense that we know there's tremendous amount of potential power in the technology, but the hard part is, well, how do you control it?

As the musician Sting once sang, "I hope the Russians love their children too."

Right?

- The future for AI research here at the test pilot school with the X-62 and what it supports in the future is really exciting.

We're going to be looking at AI algorithms that can learn.

I mean these are really important research questions because there are certain behaviors that are very human as these things learn and there are other behaviors that aren't very human, so it's very important be able to interrogate that.

How do these algorithms develop as we train them, as we put them in situations that are slightly different than the ones that they were trained for?

X-62 as a research platform is a huge enabler for all kinds of other research.

One of the very exciting projects is the XQ-58A Valkyrie down at Eglin.

We flew the first version of its AI machine learning algorithms here on the X-62 that will now go out to the Valkyrie and they're going to continue on that platform to mature what they originally began researching here.

In the same way some of the dogfighting algorithms that we have built out here are going to move on to a project out at Eglin Air Force Base focused on multiple F-16s flying autonomously against each other and they're going to use even more advanced sensors and even more capabilities to do that discipline exploration of the unknown that lets us find where can we trust these algorithms and where do they still need more development for us to really be able to say, yes, they're safe.

Yes, they're effective.

Yes, they're efficient.

And above all, yes, are they ethical?

Our hope is also that in partnership with other government agencies like NASA, that we can help develop the frameworks and the ways of thinking about autonomy that make it possible for civil aviation to begin to explore different pathways for certifying AI in applications beyond the military.

- It's you're riding the bicycle slowly, holding hands of it, and eventually you let the bicycle go.

We are finally at that point, I think, for some of them, letting the hands off the bicycle go, but that's not gonna be good enough.

We have AI systems now that can perform very well within certain parameters, but when you go outside that environment when for example it's a thunderstorm something where you haven't been able to train, then the system's no longer reliable and it doesn't adapt and adjust yet the way a human would because we don't have what we call artificial general intelligence.

We have only the artificial narrow intelligence that can do a specific task.

And the problem is that the number of different ways that a situation can be complicated, well, for a car, it's one thing for an aircraft with three dimensions is another thing now put aircraft in battle with enemies moving at you and the electromagnetic spectrum, the dimensionality multiplies and if it becomes almost impossible to train for every circumstance.

I'm one of the skeptics that believes that the current generation of AI machine learning, although we have a lot of ways to go, we can start to see the limits to where it can get to and probably another revolution beyond that will be necessary to get us to a place where we can really do the sort of advanced AI that we're dreaming of now.

The interesting thing about AI and AI empowered weaponry is that what happens is really going to evolve over the next year or two.

What we do at this juncture really makes a difference for the development of this weaponry.

The inherent concern is that it's something that we can't even sit here and speculate on, that something happens that's beyond our, you know, realm of imagination and we've already empowered these systems beyond the point to get them back.

- Critics would say that there are concerns with the potential of things going wrong with AI taking over.

- We have to design very carefully and we have to test very thoroughly to ensure that we don't have something like that happen.

We also have to make sure that the other side can't somehow take over control of these aircraft.

So we've gotta design all that in.

And one of the things that has surprised me a little bit, quite frankly, is how our fighter pilots have embraced this.

They recognize that a modern battlefield is gonna be an intensely lethal place no matter what, and they would much rather have some of these uncrewed aircraft out in front taking fire so that they don't have to.

This is gonna save a lot of pilots lives when we get this implemented.

Getting meaningful military capability fielded is what I care about the most and I care about our pacing challenge, China, the most as an adversary.

China has been working very hard, and this goes back, oh, at least 20 years, to field the military that can go head to head with the United States.

And first if they can prevent us from intervening in the Western Pacific and then defeating us if we do.

They've filled in a number of systems that are designed to take on particularly our Space and our Air Force or airborne assets, wherever they may come from, and we've gotta respond to that.

They're moving forward aggressively and I expect that the disparity in time between when we can field this kind of capability and when they have something comparable is not gonna be very much whichever way it comes out.

- How do you think the role of the fighter pilot is gonna change over the years?

- So I think there's still gonna be fighter pilots in combat air crew for a number of years.

We're still just broaching the capability of what AI can handle, and we are taking those, you know, safe, ethical steps to make sure that it behaves as we expect it to behave before we really use it in a more wide sense.

- So technically integrating AI into older systems, those are difficult challenges, but they're all doable.

They're not really unique to AI.

I think one of the real difficulties of AI is on the human side.

People are uncertain whether you trust it.

I think we still are at the beginning of AI.

I think what AI can do today will even in three or four years look so mundane, we won't even call it AI anymore.

- As AI piloted aircraft are beginning to take flight, how we handle ethical, operational, and technological challenges will set the stage for this next chapter in aviation history.

We'll see you next time on Behind the Wings.

(upbeat music) (upbeat music continues) (music slowly fades)