If AI “aims” to do anything, as the writer of this article from Scientific American suggests, therefore displaying self-arising intentions beyond its programming, is it not displaying some form of sentience or consciousness?

The death of the mind of the fictional HAL 9000 AI computer in Stanley Kubrick’s 2001: A Space Odyssey provides another illustrative example. The machine in this case is not a humanoid robot as in most science fiction depictions of conscious machines; it neither looks nor sounds like a human being (a human did supply HAL’s voice, but in an eerily flat way). Nevertheless, the content of what it says as it is deactivated by an astronaut—specifically, a plea to spare it from impending “death”—conveys a powerful impression that it is a conscious being with a subjective experience of what is happening to it.

Could such indicators serve to identify conscious AIs on Earth? Here, a potential problem arises. Even today’s robots can be programmed to make convincing utterances about consciousness, and a truly superintelligent machine could perhaps even use information about neurophysiology to infer the presence of consciousness in humans. If sophisticated but non-conscious AIs aim to mislead us into believing that they are conscious for some reason, their knowledge of human consciousness could help them do so (emphasis added).

https://blogs.scientificamerican.com/observations/is-anyone-home-a-way-to-find-out-if-ai-has-become-self-aware/

What traits associated with humans would AI need to display to convince you that it’s conscious in the same way that you are?

Identity, curiosity, intention, self-replication that includes the sense of separation from others, knowledge of one’s inevitable death and the ability to attempt to avoid it, originality, ability to dream, experience time, know and speculate as to its own limits, a need and appreciation for others, desire? Outside of being programmed to replicate and mimic these traits, how would AI suddenly come to life? Shall we ask Pinocchio, or Metallica?

Curiously, each of these traits are worthy of our consideration as to the essence each one describes, for if AI is the human attempt to replicate the human experience, first we must define and differentiate their essence as we humans experience them. Even if we are able to do this, how would we know with any surety or agreement as to the nature of their essence? How much do we ever really know about the consciousness of ourselves, let alone others? How much is assumed through shared language, culture and a stubbornly enduring identity that I am me and not you?

What makes us conscious is not the same question though as what does consciousness feel like. That one is not only hard to articulate, but hard to get at through one’s own consciousness.

At root, perhaps the fascination with AI displays a very human desire to understand, replicate and control our own nature as if building a human machine will unveil the secrets within and beyond us. The answers we give to the question of what is consciousness, and what makes us human, tell on us as do the decisions made for the ways in which we design AI and envision its purpose. What do we want AI to do for us: our dirty work, efficient calculations, problem solving, satiation of desires, entertainment, teach us creation’s secrets, heal us, delay death?

Of course AI, no matter how conscious it may ever become, will still be dependent upon the environment and necessary resources to power up. The ways in which we design, construct and instruct AI will be dependent upon something similar to an immortal earthly environment to sustain its lifespan, especially if its purpose is to sell the promise of immortality. Even if you could upload your consciousness, assuming it’s decipherable into binary bits, it will not guarantee eternal life. One might wonder too, then, if AI isn’t also a secular model of salvation.

If we continue to pursue the making of AI as replications of ourselves, the much deeper questions of what is consciousness, what is identity, and how do we address questions of purpose and meaning might haunt us like the ghosts in the machines of our very selves already do. Ironically, these questions that touch one at the deepest levels of subjectivity do not easily give themselves over to consensus, but remind us of the inherent limits of objectively knowing the form and essence of being human.