Artificial Intelligence Is not “Intelligence” — Benjamin Harris

One of the things they have to do is write down, "Here's the rules of the game, and here's what, uh, doing a certain good thing pays you," right? They call it a payoff function. And so if you, you know, if you take the opponent's pawn, little piece, here's the payoff. Or if you take their queen, it's a bigger payoff. And so, um, if you can define that—in chess you can actually define that—you can make a system that that does very well. But think about a dating relationship or a marriage. Like, and we pretty quickly realize the—our attempt to quantify and define value, it's—we're in a different layer of abstraction. We—you can't—they—these—these don't go together.

So today on Anabaptist Perspectives, I'm joined by Ben Harris, and we're going to be diving into the ontological limits of artificial intelligence. Um, Ben, you want to start with a little introduction, and we'll jump into the topic after that.

Sure, Marlon. Good to be with you. So my name is Ben Harris. Hi, everyone. So I'm a professor up at Sattler College in Boston, Massachusetts. I coordinate the business program up here, but my—my background comes out of the engineering world. I spent more than a decade working in machine learning, artificial intelligence, this—many of the classical engineering disciplines. So, uh, I would—I think it's hard to define what an expert in the field is, but, uh, I spent a long time thinking about it and continue to do so because it affects not only back in—in the technical world, but also in academia. We contend with AI every day. So, Marlon, it's good to be with you.

Yes. Thanks for coming on, Ben. Excited to have that kind of engineering computer background that you bring to it. So, to start with some of the drama around AI, um, a little over a year ago, uh, May of 2023, um, there's this famous statement on AI risk, um, signed by, you know, a bunch of the big names.

And their short version was, "Mitigating the risk of extinction from AI should be a global priority alongside other societal scale risks such as pandemics and nuclear war." And so that got a bunch of a press. I found it interesting. A few weeks later, there was a CNN report on a survey from a number of CEOs.

They got 119 responses, and out of those, 50 of them said, "Yeah, AI could potentially destroy humanity in the next 5 to 10 years." And the other 69 were unconcerned. So I guess we could start with where are you at on that question?

Your view of artificial intelligence and that kind of, yeah, that kind of dramatic fear statements. Oh, that's a—that's a classic question for anybody who thinks about AI. I—I—I am not on the what we call an AI doomer. Those are—those are probably what you'd put those, you know, uh, the Jeffrey Hintons of the world that would, you know, he's very—he's a much longer background here. This is his view. I do not see AI as, within that short period of time, 5 to 10 years, as being an existential threat to humanity, um, both for a theological reason, I think, like God has defined the end of humanity for us, but also that we—we—we look at the technology and what we assume in the forecast of, you know, AI improving or growing to a certain level assumes no major hurdles that we encounter on the way.

I just—and I just think we're starting to see the hurdles, right? We—we at the moment cannot produce data fast enough in order to support these bigger and bigger AI models. We don't—we're running out of, um, raw materials for creating training chips and systems that do this. So there—so we're starting to see the hurdles, and I think that that's going to—that's going to skew the forecast fairly dramatically. Um, I—I am not particularly concerned about existential issues, um, partly because what we're going to talk about today is the—is the ontology of AI, right? The—the—the movie versions of AI being a threat to humans, um, in the—in the narrative tend to revolve around a moment when AI realized what it was and what it could be, and it has this self-defensive response or reaction to humanity trying to shut it down.

I don't yet think that AI has the—has an existential understanding of what it is yet. You know, it—it probably can give you a text response, "Yes, I system, system, system, and an—and an AI system," but that doesn't—that does not yet imbue value in a self-defensive reaction, right? That—that we naturally experience as humans. So I think there's still a gap ontologically, and as well as what we would call artificial generalized intelligence or AGI, right? Systems that can think and reason for themselves. Um, I mean, it's still an ongoing open research area in—in what they call argument mining, right? You can present a bit of text to an engine and say, "Is this a good argument?" And it has a really difficult time determining yes or no.

Whereas we as humans read that and say, "Oh, that's a terrible argument," or a great argument, and I find it very persuasive. And so the—there's—that's one I think about a lot, that particular gap. But there are a number of them that—that in my view set—they set a fairly big chasm today between where AI is and anything that we would need to worry about from an existential standpoint.

So you're setting a chasm, and you're saying those time frames are way overinflated, or sorry, not overinflated, opposite of overinflated, way too ambitious in a sense. Um, maybe to push on a little bit, I still hear you using words like "not yet."

Um, is that implying that the time is coming? It's just not yet. Um, and I guess even to push on that a little bit more, you know, we talk about things in computing like Moore's Law, which I think is a fairly specific technical definition for that, but in a general sense, you know, computing power changes very quickly.

Um, I mean, you even noted in an email to me, it seems like there's new AI applications coming out every couple of weeks. Um, so I guess, how would you respond to an argument like that that says, "Well, you know, computing power has been doubling frequently for the last number of years, and we're just going to see, you know, we can't see what's around the corner"? Yeah, it's a fair question. You know, why—why the not yet? I think the—the one of the reasons is if you, you know, say you—you project out some number of years and you—you allow Moore's Law, which is beginning to show asymptotic behavior, right? You—if you were not doubling the, you know, our—our speeds are getting quicker, but we're running into computation issues with these kind of things.

That's why largely why GPUs have become the—the standard architecture for doing this kind of work. One, the not yet is an admission that I don't—that we don't know, right? We, you know, can—can you create a technical system that—that is a—a version of intelligence? That's a bigger question than I think AI can answer. That's a, you know, what is—what does God define intelligence as? And can you create a system that has that? Um, you know, and we—we sort of think of it in a—in a narrow band in that, you know, intelligence is the ability to collect and synthesize information to answer a question or to develop something. Um, but there's—there's other types, right? We have a sense of aesthetics, right?

Humans look at what is beautiful. We have a—a sense of awe, um, that is uniquely human, right? And you, you know, if you ask a—a large language model, you know, if a given painting is, um, is beautiful or if a piece of artwork is just abhorrent to you, it—it likely does not have an opinion. And if it does, it has to create it based on some quantitative aspect of the artwork. It has to look at, you know, the—okay, the color contrast is within this range, and most people think that that means it's not a very attractive painting. It has—that's the way it has to think because of its computational nature. Um, we don't think that way, right?

We—we look at it and what does it—what does it do to our—to our spirit, to our soul, to our mind? And we have a response. And so, um, you know, I think that the not yet is a—you know, is a—is a fair question. I think we're—we're also running into some mathematical challenges in that the—the number of, um, I'll call them synapses or nodes that we'd have to do to simulate actual human thought and cognition is still many orders of magnitude above what the most powerful systems can handle right now. So even if Moore's Law holds, which is a question of whether it will, we have a long time to go before we exponentially reach that point we would need to be.

Um, and who—and who knows, on the way, we may encounter a whole nother obstacle we didn't anticipate. So, um, the—the challenge is that we—you don't—when you make a forecast, you have to acknowledge what could cause the forecast to fail, right? Things—things don't always continue as they were. You look at, you know, population explosion in the 1970s, right? There was this huge concern that it was going to lead to global famine and billions of deaths, but that never happened. And so, um, so we want to look back soberly with—at those examples and say, I think with a bit of wisdom, um, and just encourage ourselves that, like, the Lord has this in hand, right? He's not—he's not going to let AI ruin his creation.

Okay, so a theological answer there, confidence in God. Um, yeah. And the title, Ontological Limits, kind of highlights what I wanted to—the question I kind of wanted to press here. Um, you know, if we talk about limits of AI, um, maybe the first limit we think about is technological. Can we build it? What can we build? What kind of computers can we build? What kinds of neural networks and so on?

Um, or epistemological, what do we know how to do? Knowledge. Um, when we say ontological limits, we're getting into some ways the strongest of those terms, because we're trying to say, what—what is the limit by the very—by the very nature of what this thing is, um, that produces the limit? Um, and yeah, I think maybe you've hinted at some of that. Um, but yeah, kind of at its core, what you thought see as that biggest core limit or way of talking about that core limit? I think probably the—the—the most stark ontological limit of AI is—is just what we've created it out of, right? This is a—this is a creation of, um, well, it's—it's a—it's not funny.

Our creation of creation, right? We've—our—our ingenuity has developed this, and it's immensely powerful in some areas, right? I don't want to—to minimize its effectiveness and its aid in some things, but there, um, it is a—it does not involve the supernatural, right? It does not, you know, it is a—it is a by definition natural development. It is limited by the laws that—that God has put in place in terms of physics and computation and—and information, uh, and mathematics, right? We look at, you know, questions like the—there's a famous question called the halting problem in computer science that basically is a—we now know is an unsolvable question for a computer system. And so there, you know, it's not without—computing is not a—a finished business, right?

That we, you know, we know how to do everything as long as we have enough power, enough, uh, enough systems, enough electricity. Um, we don't. We—and we can prove that we don't. Not only that, but never will, right? There's—you know, mathematicians have worked on that, that there are some fundamentally unknowable things in computation in the technical fields. Um, and so I—I think those are—you're going to encounter some of those questions in the growth of AI based on its ontology, right? You can't—there is not a—uh, it is bound by the laws that—that all physical, um, digital systems are bound by, right? It can only go so fast. It can only get so hot before it breaks down.

It's limited by the laws of physics. It—it is also limited by the ingenuity in which humans can insert into it, right? We create its steps, but we are limited and finite. So I—I—I have a difficult time imagining how a system is going to, you know, by its own volition, exceed that based on what it's created on.

Well, it's part of the—the argument there, especially for the, I don't know, either AI optimist or AI pessimist, whichever one it is, that, um, you know, has these very high expectations for AI. I suppose that it's more a question of your outlook, whether that makes you an AI optimist or pessimist. Um, but it's part of the argument that, well, these are neural networks.

They're functioning the same way as the brain. We don't have to give it a precise algorithm because it has more capacities for self-learning and self-adjustment. And is the—is the expectation there that something about that architecture is going to let it get past what normally applies to other computer systems or? It's a good question. There's a—I remember there was a—there was a French institute some years ago that had begun to try and create neural networks with the scale that would attempt to approximate some of the cognition of the human brain.

And, you know, they had the—the resources of the French government. They had an immense amount of computational power behind them. And even with all that, they were able to—to roughly get to, if I remember the number correctly, so don't hold me on the—on the citation, something like 2 to 3% of brain function.

You know, this is—this immensely complex, because network complexity does not grow linearly. It grows exponentially, right? To gain—to grow a system that can get strong enough, um, to be a big enough neural network. Even—even that is—is not going to give you human cognition. Because we think about the—the use cases that we use neural networks for. There's—there's really—there's two major ones in machine learning. One is classification, basically telling, you know, you look at a—you take as input something, and your brain tells you what the thing is, right? Our brains do that naturally. You have image recognition software or other neural networks that do the same thing. They take in inputs, and they identify something.

Uh, the other one is regression. It's making a—um, a relationship between two quantities. So, uh, and allowing you to do a sort of what we call an in-sample prediction, right? If you're—if you know, um, you know, house size A and house size B, and you see something in the middle, what should the cost of the home be? Right? That's a—that's a regressive—uh, or sorry, it's an example of regression. It's not regressive. But the—um, and those are the two main neural network applications. And there's now—um, there's variations on them now. Those—the base layers of those are what's been built up to create these transformers that LLMs are built on. So, uh, they exist, and they've been extended.

But you—I would say we are—we are too far away from, um, actual human brain function to predict that it will—it will continue along that road, even—even on any, uh, uninterrupted path and finally get to how the human brain works. Things were going—there's going to be breaks or discontinuities in the progress. And that—and who knows? Some of those may scupper the whole thing, right? You may only be able to get so far with the applications of AI. You just can't go further. They estimate that the cost to train—training is what's most expensive right now for these systems. Mm-hmm. Uh, to go beyond GPT-4, uh, sub 4.0, 4.0, um, to GPT-5, GPT-6, GPT-7 is trillions of dollars per iteration to accumulate all the data, to do the training, the electricity costs to run the models.

Uh, at some point, we're going to—we're going to run out of money. We just, you know, humanity will either have to decide, okay, we're invested in this or we're not. We just can't keep spending that—those amount of resources to develop a system like this. So I don't—you know, fortunately, I don't have to make—I'm glad I'm not in charge of an AI company that I'd have to make that choice. But, um, there's—you know, whether it's the actual technology, whether it's the mathematics behind it, or the funding to generate these things, uh, any—any combination of those can cause this thing to fail. So we have to almost have the perfect storm to go up the—the graph of progress towards human cognition, at least in my view.

Yeah, those are helpful. I mean, one, the point you made there at the end of literal physical constraints. I mean, we have all kinds of energy available, but that as you try to exponentially scale up the amount of energy, you're literally running into, I don't know what the scale is, but it's registering on things like grid capacity and electricity generation capacity and that kind of thing at some point. Mm-hmm. Not a small thing. Um, no, the other thing that was really helpful, um, for me and what you said there was AI is really only doing at the base the two operations, um, classification, classifying objects, and it's gotten sophisticated at that by—by training, humans coding examples, um, and then it going off of those examples and regression problems, which I'm not—not familiar with, um, the mathematics of those.

But again, those are fairly simple operations. I mean, it takes a lot of power to carry them out. Um, fairly simple compared to the human mind and living a human life. Um, so yeah, I think just breaking it down and saying it does these two things and builds on them and puts them to good use, um, to me really helps to see the a little bit of see the limits or demystify things a little bit.

Yeah. And I—one thing—one thing I would add in too is that we—you see on, um, maybe on social media at points, people, some very like AI negative people who are like, "AI is not any good at anything." And they'll bring up an example. Like I saw one on LinkedIn the other day that there was a, um, there was like a stone pillar in a field, and on one side of it had like the rear half of a cow. And then the stone pillar was maybe 10 feet wide, and on the other end of the stone pillar was like the head of a cow poking out. And now we would say like, "Okay, there's two cows in the picture."

But the—like an AI system might put a box around it and say, "Here's one cow, and its length is 15 feet." Right? And so they would look at an example like that, and we know humans know the error. We—we see it, and we're like, "Oh, okay." Uh, a computer system doesn't. And so I think those—those examples get brought up to say, like, "AI is of no—like, it's never going to be a threat." It's—um, but I think what it points to is there are—there are—there are limits and easy mistakes that AI makes, especially early ChatGPT days made tons of mistakes.

And so, um, but humans will—will continue to try and sort of plug the holes in the dam, right? They'll—we'll say, "Okay, here's a—here's an error that we're getting ridiculed for this thing that the system can't do. We'll fix the thing." Uh, and I'm—I'm led to believe we're going to keep finding more of those mistakes that humans have to fix. And it'll—it may get better and better and better, but it—again, you're—you're going to keep finding, uh, reasons—reasons to make fun of it, ultimately. Now, that doesn't mean it's not powerful. It's not something to think about, but it's, you know, those—I think the—the—the satirical view of AI is becoming more and more prevalent. And I don't know what that's going to do to people's view of it.

Um, probably very little. But, um, we won't. I just want to—I want to try and think clearly about, is this a—like, what is the truth here about AI? Yes, that's a silly mistake that it made. But does that invalidate the whole project? Well, of course not, right? It's, you know, amongst the billions of things it can do, here's one that you thought was silly, right? So move on.

Yeah, exactly. Um, I mean, just to pursue that a little bit, though, I mean, yeah, I—I like the case you've made on kind of, um, limits and it's limited. Um, but would you agree that we can get—we can get a lot better at using some of the tools even without—even without really fundamental changes in capacity? Um, like, for example, I was listening to an accountant, and his statement was, "It's going to come a time sooner or later where your basic bookkeeping, categorizing transactions and so on, they're going to—somebody's going to come up with a tool that does that well enough that it's barely worth your time to have a human bookkeeper go through in detail because it's going to get it close enough, especially for the purpose of a small business.

It's going to get it close enough that it's not going to matter." And I don't know, that kind of strikes me as plausible. It also doesn't strike me as requiring anything radically new from AI, maybe just some human ingenuity and how to—how to apply it right to the process. Um, I don't know. Does that sound like a fair estimate?

Oh, I think so. I think there's like accounting as an application of AI. Sure, you can do the—the mechanics are not tremendously complicated. They take care and some background knowledge, but the—uh, nothing about the mathematics is incredibly difficult. But the—um, think way like with that example, one of the constraints we're going to encounter is at the end of it, that you—you make a legally binding declaration, right? These are the—these are the truth of the accounts. And so who—who is going to—to put their legal weight behind that? Is it going to be the—you know, is it going to be the individual who's used the system? Like, I certify this is correct, or are we going to try and pass that off to the AI system?

So like, well, no, the AI did it. So if there's a mistake, it isn't my fault. It's the AI's fault. And if we do that, which AI companies are going to shoulder the legal burden for that? Of, you know, that's—I think it's—it's going to sort of be the—the—the fingers pointed at one another issue with why hasn't this become—why hasn't this use case become widespread? I think it's because no one is confident enough yet that—and I'll say yet—because I think it—it may come a day that people will be and this will—the dam will break loose. But I don't think anyone yet is ready to sign up and put their legal business life behind this. I think people are still waiting for—for more and more evidence that it's okay.

Yeah, and that could apply to—to a lot of pieces where you have software doing a lot of it. I mean, just thinking, you know, friends that build roof trusses. And this isn't necessarily AI, though, but the software is doing basically the engineering and saying, yes, this—this truss will meet specs or it won't. Um, but if they're actually doing a job that requires an engineer's stamp, well, it's still got to be an engineer that does it, which is an extra step, that liability part. Mm-hmm.

There. So good. Yeah. Okay. So I'm guessing, given the arguments you've made that, you know, artificial general intelligence, um, actual purposeful behavior by AI systems and so on, um, I'm taking it you're very skeptical of that. You're very skeptical of that being in the offerings.

Yeah, I'd say that's true. I have a good degree of skepticism about that. The—I think that we have—humans have struggled to define what AGI actually is, right? How do you test it? How do you verify it? How do you—and so I think that that particular question of what is AGI is going to, um, remain in the academic circles for a while, right? It's going to, you know, we're going to—there's going to be arguments for and against of various kinds that, um, may or may not prove fruitful, right? I don't know that they're going to come to any conclusion. Um, and I think in the background, AI companies are going to continue to build systems that more and more closely approximate, um, you know, human cognition, even though they—we—we might say you're still light years away, but we're making progress, and that's probably true.

And so, um, so I don't—the—yeah, I'm not tremendously worried about the development of AGI, um, you know, and sometimes the—the cases of chess or Go, these board games that have had a long and storied history in computation, um, like, uh, companies like DeepMind, so, uh, which is now owned by Google, but that's out in London, uh, have done some really neat work, like developing superhuman chess engines and Go engines that play the game at a—at a level that surpasses human understanding, um, which is pretty cool the way what they had to develop to do that. Um, but those are still games with a—a defined feature space and defined rules and defined options, right? You, like, when you can do—when you can do that, when you can box in the system, the universe, you can make something really neat.

But boy, humanity resists being boxed in like that. We just don't. It's not our nature. And—and so I think part of that is the root of my skepticism is, um, like, if you look into a field called reinforcement learning, it's a—it's a subpart of machine learning, but it—one of the—to do it well, this is how the—the, um, the chess engine from DeepMind was built. Uh, one of the things they have to do is write down, here's the rules of the game, and here's what, uh, doing a certain good thing pays you, right? They call it a payoff function. And so if you—you know, if you take the opponent's pawn, little piece, here's the payoff. Or if you take their queen, it's a bigger payoff.

And so, um, if you can define that in chess, you can actually define that. You can make a system that—that does very well. But think about a—a dating relationship or a marriage. Like, and we pretty quickly realize the—our—our attempt to quantify and define value, it's—we're in a different layer of abstraction. We—you can't—they—these—these don't go together. And so, uh, you know, I—those are some of the, I guess, musings of, to me, of why I think AGI is still some ways off, if we even can define it.

Yeah. So along the lines of, you know, AGI and again, these questions about intelligence, um, the Turing test, I used to use a version of this with students, um, long before, you know, generative AI was on the—on the horizons.

Came from my—my days in college, philosophy 101, and, you know, basically we're asked to—I don't have a technical definition in front of me, but we're asked to consider the question, you know, if a computer can give the same answers so that if you're communicating with it through text or whatever, you can't tell if there's a computer or a person on the other end, um, well, then should we ascribe it the same mental life and intelligence that we ascribe to a person? Um, yeah, how do you think about that? And maybe you have a more precise, uh, computer science definition of the test.

I don't know. No, the—the Turing test, yeah, it was all the way back in the 1950s, Alan Turing, uh, when he was working in the early theory of computation. Um, no, you—you had a—a good working definition. You know, if you're—if you know there's two rooms behind you and you're getting text inputs from questions you ask, how do you tell if one is a human and one's a computer? And if it's, you know, a system that is statistically indistinguishable from a human, we would say, okay, passes the Turing test. I think we are actually already there with that, with the Turing test. I think we have systems that you can—um, you can write fairly complex questions to, uh, and this is always an interesting, like, thought experiment of, like, what question would you ask that an artificial intelligence system that you know is an AI system would fail to answer well, right?

Um, that's kind of an interesting—sort of a subfield here. But I think we're—I think we already have systems that pass the Turing test. The—uh, but I think this is—it's the current application of the Turing test, I think, is a moving—um, a bit of a moving standard in that you—you have a system that seems to pass the Turing test, and then humans get used to how it communicates, right? You begin to pick up the flavor or the nuance of, like, this sounds like it was AI generated, right? I think we all, if we've read AI generated text, we sort of know what that feels like. It's very—it's very linear, very clear. There's no halting, pausing. There's—it's like, it's mechanical in a sense.

And so, um, now what an AI company might say is, okay, I'll adjust my—my generation algorithm to make it a little more clunky, a little more human-like, right? And then it—it passes the new standard of the Turing test that people can't distinguish. Um, but I—whenever you have a test that relies on human perception of something, that, uh, like, we grow, we learn, right? That—that the benchmark for that test is going to change over time. So, you know, I think we—we have systems that at early GPTs probably passed the Turing test. Now the standard's even higher, right? For a system to be indistinguishable from a human. Um, and it's not because the systems have changed so much as we have changed and learned.

And so we've learned how to—how to deal with it. Yeah, that's a good perspective. And just, yeah, as we're talking about this, and it occurs to me that it'd be very interesting to read what philosophers wrote about the Turing test and how that has changed as the computer capacities have gone up. Because that was the angle from which I came, which—which I would have encountered it was kind of philosophy of mind and what is it about the human mind, what makes mind, and so on.

And it does strike me that those kind of questions are really kind of relevant to how we're thinking about AI here because if you're coming into the whole discussion, um, with a philosophy that says, you know, we can explain this all physically or we can explain this by the functions of physical things, the human mind is just—just is the human brain or it's the functions and processes that the human brain runs, then it seems to be a fair question. Well, can we duplicate it with something else?

Um, now if we're coming in as Christians, we still may have a large variety of philosophy of mind. We're going to believe that the brain is important. Um, but generally, you know, God is a spirit. God has a mind without having a brain.

Um, generally, we believe that our mind is, in a certain sense, independent of our brain, although obviously it functions through our brain. Um, yeah, I just have to wonder how much that plays into the whole discussion. Um, if you simply are a materialist, then it seems like, you know, kind of naturally lead to this more AI optimistic view of things. Um, I don't know. I don't know how that maps into the academic landscape either.

There's—there's certainly in the academic world, there's certainly a sense of techno-optimism. And it's, uh, and they, you know, they treat it, again, if you—if you come at it from a largely materialist worldview, you end up with just a, uh, you know, an understanding that that all I'm—all I'm creating technically is a smaller approximation to me, and it's going to get closer and closer and closer over time. And so there's no—there isn't really a connection between soul, spirit, and mind in the—in the secularist worldview because we're just, um, you know, we're just a random collection of atoms that that have no lasting eternal value. I think that's the conclusion. Um, whereas I think, like, that is from a Christian perspective, that's where we come at AI with a different view of, like, you know, yes, say you can create a neural network that is—that has the trillions of connections that our brain does, um, you're still missing a piece that you can't simulate.

There's a spirit, there's a soul, there's a—humans are, you know, by a mathematical definition, irrational creatures. But in—if we—because we are, we, you know, we look at—we do things that are silly and not in our best interest. Um, but in—from a theological perspective, that makes sense because we—we identify this conflict of our—of our sin nature and our redeemed nature in Christ. And so if you don't have theology, this gets—leads to a natural techno-optimism, but I think only in the—in the sense that it is self-beneficial, right? Like, I'm optimistic technically if it benefits me, right? Now, if it doesn't, if it's a threat to me, then I'm not as optimistic about it. And I have no moral qualms either way, right?

That's the materialist view. It's not a moral question. It's—it's pragmatic. Yeah. So actually, while I was getting ready for this episode, I was sitting in my little, uh, office at the school I help with, and I look out the window and, yeah, I notice plants, flowers. It's a beautiful sunny day and butterflies flitting around on top of there.

And, yeah, so I'm drawn into the beauty, something a lot of people are. And so it just got me thinking, you know, those are the things we paint, uh, we draw pictures of. Um, but not only that, like, we've really have—we really have dug into that. What is the life cycle of the flower? How can we breed the flower to maximize blooms?

You know, we've traced the life cycle of those butterflies and learned how they function, uh, learned the biology of how they function, the life cycle, all of that. And, you know, I even had to think about a book we're using in our—in our science program. It's called The Girl Who Drew Butterflies.

And it tells the story of a girl who was a painter, her art, um, but just was really—was really closely observing, you know, insects, butterflies, came to understand these stages of metamorphosis and so on, um, when a lot of people—at a time when a lot of people didn't understand those. And so it just kind of really hit me, like, okay, this is what humans do with things that we see and pay attention to.

Um, is there any reason to think that AI models or agents, um, would do any of the same thing, would have any of that same—really, it's a relationship, ongoing relationship with reality. Um, and does that get at any of the difference, this kind of difference we're talking about between a human mind and an AI model? Yeah, love to hear some of your thoughts on that.

I love the question. I, I, um, I mean, my—my thoughts first go to the—the—the genesis of what we would call curiosity, right? The—the, you know, what I have—I have children at home, some of whom are very curious. And what is it that—and we love curiosity. It's how we—we develop an ever-expanding knowledge of the world, um, you know, on—so on the one hand, I think AI could be constrained in that it has to, like, it doesn't really have the ability to explore aside from the digital space it inhabits. And so, you know, it could, but, you know, that could be—that could be overcome if a, you know, a human—we—we give it inputs from a world beyond its own, right?

We give it pictures and soil samples and scientific reports, and we give it enough information to discern connections and patterns and systems. Um, but I think at the—at the very bottom of a—of a contrived system like that, we have like human volition. We've done it. We've—we—we have to push AI into the world to begin to collect this information. Um, so I don't know that, um, you know, I'll say this: in the—in the world AI inhabits, I think it already is acting as a curious agent within its world, right? I think by its incentive structure, right? It says, like, I want to be the—the best AI system, so I'm going to collect everything that I can, um, and and look for patterns and and begin to create more and more of an approximation to reality.

Um, there's so much more to the world than just the digital space, right? So we have to, you know, you have to somehow connect AI with that space. That's a more challenging question. So I don't know that I would—I wouldn't—I don't think that AI is going to have agency in that way. I think one of the problems we're—like, I think we're already seeing an inflection point with that kind of pattern in AI, um, because they estimate the amount of—the amount of data on the planet doubles every six months, something like that. So just the amount of data is exploding based on what we're creating: movies, films, text, all these things. Um, now what we're realizing is that some of the data that's being fed back into these models to learn and remain current is actually AI generated, right?

So AI models are consuming AI generated things and taking it as—as new truth sources, right? So they're—they're—they're building up on what they—what they've already created. And so any, um, you know, if there's any error or bias or anything that's built into those generated sets of data, that's now been replicated and virally expanded across the AI space. And so, um, so I think, you know, my one—one thought I have is there's going to be a bit of a pendulum swing when we realize AI is spitting out nonsense because that's all it knows. Um, and the pendulum swing will say, well, don't—don't let AI have access to anything new yet, right? We—we need to curate what it's learning from, um, in order to to make this truthful.

And so, um, you know, we have this curious—humans have this beautiful curiosity to know, you know, what—why do things work the way that they do, right? It's led to advances in cosmology, into biology, in mathematics. You know, we do this naturally. Um, now I loved—I loved seeing recently there was a paper that came out of, um, I think it might have been Google DeepMind as well, but they actually had developed an AI-generated mathematical theorem-proving system. They could give it a mathematical question. It could prove something mathematically. I said, that's fascinating because that's—that's an area that I had thought that humans, like, that was human, like, creativity and intuition. That was not an accessible region. Um, but I had—but the more I am reading about it, the more I'm thinking, okay, they've not done human things, but they've—they've made progress, right?

They've done some things that I thought they couldn't do. And so, so they're—but I think that's how progress oftentimes goes in AI is they—they—it appears they can't do anything, and then there's this discontinuity and jump into now what they can do. So I think we've seen those now. Do I think that that AI is ever going to have the the beautiful, uh, observation of the butterflies, like the girl who drew butterflies, to to understand the biological cycles of these things?

Again, I probably hold a healthy skepticism, just like AGI. I'm not—I don't think that AI has—because we haven't given it the reason to yet. We've not incentivized it. Um, you know, we—we learn because it's beautiful to learn. And AI learns because either we tell it to or we threaten its extinction if it doesn't. So there's a, um, you know, I don't think we—it doesn't have the natural curiosity we do. And I—and I think that makes us human.

Yeah, thanks. Yeah. So I keep hearing you come back to the not yets a little bit. But if there is one theme in your limits that you talk about, kind of ontological limits of AI, is—is it always being downstream of humanity in some way or another? Yeah, we don't understand exactly what it's doing. We—we set it up, and we don't always understand exactly how it works, the whole black box effect. But it's downstream from—you're talking about the incentives we give it, the way we program it, the parameters, what we want it to do, the inputs we curate for it. Um, so yeah, that is helpful. Um, yeah, any particular concluding thoughts you'd like with—like to, uh, leave on this topic?

I think just the, you know, your—your right point to highlight, like, yes, I—there is a—there is a sense of not yet. And part of—again, part of me, I go there because I, in candor, I don't actually know, right? I make a forecast that could have a, you know, there's a margin of error in the forecast, and there's things that could go wrong or right that make it wrong. And so, um, I'll only forecast out as sort of as far as I could see. The, um, but you think about something like quantum computing, so a new technology that there's engineering challenges to solve, but say we solve them and we have quantum computers that can do incredibly quick calculation in—in some previously inaccessible problems.

All of these developments, you know, AI developments, growth in neural networks or training or, you know, GPT-8, say, or quantum computing, right? They all solve sort of little niche things, and they do it very well. But I guess the analogy I keep—I go back to in my own mind is that you, without a—without a unified system or understanding of reality, all of these things will remain niche solutions, right? You don't—it'd be a little bit like, you know, you're trying to fix something wrong with your body and you, you know, you—you sprain your finger and so you splint it, right? You have a solution to the problem. You've not cured cancer, right? You've—you've solved a problem, right?

And you've made progress towards the solution of the whole, but you—there is not a systemic understanding yet. And so all of these developments, um, you know, I'll—I'll say not yet. My—my prediction is it's a long way off, but, um, I think for all of these things to work in concert to develop anything that might be an ontological break for artificial intelligence, just this—to me, there's still too many unanswered questions is they don't—none of these things see reality in the same way. Like quantum and classical computing also don't see reality in the same way. So, like, you got to find a way to bridge that before these are affected together. And it's—I think that there's—we're finding more questions than we have answers to these days, in at least in my view.

That's an interesting way maybe to put your not yet. You said we won't get to an—you know, you don't think we'll get to an ontological breakthrough. Um, and so in some ways, what you're saying is, look, the ontology of what we have now is—is in reality nothing like artificial general intelligence. You can't conclusively rule out that at some point we would be able to develop systems that are really fundamentally different in how they work. And that's kind of the not yet. And as part of your emphasis is, like, it wouldn't just be a development. It would be something—would be a very fundamental breakthrough.

Yeah, I'd say that's a good—that's a good assessment of it. It's—it's very hard to prove a negative, right? So that's one of the challenges we run into in logic. True enough. So, and it, um, so yeah, I think we have—there's—and it always seems, um, it always seems impossible to solve some of these problems until someone solves it, right? There's—it's true of mathematics, right? We have all these unsolved problems until someone, um, you know, goes at a mathematical problem long enough and they come up with a solution. So I think, you know, my—my skepticism lies in the—in the—the nature of the questions being asked in that, you know, we're saying, you know, in order to create things that make us uniquely human, we needed, you know, more and more silicon chips doesn't get us there, right?

It needs to be something else, right? Some other way of understanding reality, processing information, um, that we just don't have. And I, um, and again, I don't think too long about what it would take to do that, but the, uh, when we reach a point in innovation that every time we innovate, we get more questions. Uh, I'm—I'm waiting for the—the I'm not really waiting, but in a sense, if the tide began to turn and the—we began to answer a lot of these things, or major breakthroughs, we're covering lots of them, that might tilt the needle of optimism versus pessimism for me. I might say, okay, I'm—maybe I was in error before, but at the moment, we just keep getting more questions.