This will be a little confusing in the beginning, but nevertheless i like to tell the story that way and i hope you enjoy it.
Pygmalion - the beginning
As many other great stories this epic begins in ancient Greece. Ovid in his Metamorphoses tells the story of Pygmalion for the first time. A young gifted craftsmen can't seem to find a suitable mate among the human crowd, so he starts sculpting and eventually he sculpts his dream girl in marble. Later on due to divine intervention - blessed be Aphrodite the goddess of love - his marble girl friend comes to life, they fall in love and get a child.
Pygmalion - modern times
The story of Pygmalion was retold and reinterpreted over the millennia by many many great artists. Here i'd like to highlight one of these reinterpretations.
George Bernard Shaws play Pygmalion, in which a phonetics professor teaches a young girl of low social stand and strong low class accent how to act and speak like a member of the upper class. Over the course of the play the girl participates in events of the british nobility and rubs elbows with the big ones. And thus she gains influence and the act of being high class becomes reality, as she surpasses her mentor in social class. The name of this girl is Eliza.
Pygmalion - nerdy version
Now we enter the domain of computer science. In 1966 the name Eliza - inspired by the above mentioned play - was chosen by Joseph Weizenbaum to be the name of his most famous work - the first chatbot on earth: ELIZA.
The actual code of ELIZA was in a way revolutionary at that time, because the computing logic and the actual content - the keywords and sentences - were strictly separated. Which made the program be nothing more than an elaborated lookup from a structured list of sentences. It was also very easy to just replace the content and get another ELIZA - which is the reason why ELIZAs for several languages could be developed very quickly. And honestly there is nothing much behind the code. Nowadays you could write ELIZA in less than a day. This divide between the fascinating ability of a computer having a deep meaningful conversation with a human and the sheer simplicity of the program was best described by Joseph Weizenbaum himself:
But once a particular program is unmasked, once its inner workings are explained in language sufficiently plain to induce understanding, its magic crumbles away; it stands revealed as a mere collection of procedures, each quite comprehensible. The observer says to himself "I could have written that". With that thought he moves the program in question from the shelf marked "intelligent" to that reserved for curios, fit to be discussed only with people less enlightened that he. The object of this paper is to cause just such a reevaluation of the program about to be "explained". Few programs ever needed it more.
[Weizenbaum J (1966) ELIZA: a computer program for the study of natural language communication between man and machine. Communications of the ACM 9(1): 36–45.]
However the public discourse about ELIZA focused mainly on the myth of the conversing computer than the actual demystifying triviality. A few positive examples and some misunderstood accidents around ELIZA forged the myth of the mighty psychoanalytical intelligence, which were retold over and over again and the myth became more real than the actual trivial thing. Even scientific colleagues of Weizenbaum joined the crowd with visions of an armada of ELIZAs helping every citizen in the US - or maybe they just promoted this idea to secure funding. Weizenbaum fought a losing war trying to highlight the difference between actual intelligence and the appearance of intelligence.
[If software is narrative: Joseph Weizenbaum, Artificial Intelligence, and the Biographies of ELIZA, Simone Natale]
Putting it all together
The underlying theme of all variations of this Pygmalion epic is: what makes an artificial thing real.
In the ancient version a divine intervention makes the marble statue a living being and thus it becomes real. In the play from Georg Bernhard Shaw the artificial thing is the social status of Eliza, which by imitation seemed real for a time until the illusion of nobility became as powerful as being a member of this social class.
The third Pygmalion in this article is Jospeh Weizenbaum, who insisted that his ELIZA only seems real. But his endeavors had to fail due to the divine intervention of wishful thinking of his audience. Once the myth was out it was hard to control.
Conclusion
Why am i telling this story of the three Pygmalions? First of all i find it very fascinating. And i like Stark Trek, where the topic of imitating humankind versus being human is reiterated in many central characters like Data in Enterprise, Odo on Deep Space Nine or the Doctor in Voyager.
But there is another reason. Current chatbots - or similar systems like GPT3 - are very complex. Not the algorithms, but their learned behavior realized by the parameters of deep convoluted neural nets is so complex that no human could understand it in a way that "the magic crumbles away". We are at a point were it becomes very hard to tell if a system is acting intelligent or it is just appears to be intelligent. Fortunately it seems we as mankind have grown. Current systems are much more powerful than the ELIZA of Joseph Weizenbaum, but the fascination with these systems is less enthusiastic due to frequent critical reporting about their capabilities and flaws - like latent racism. I find that reassuring, but still it makes me think what is the threshold a system has to pass to become "real" instead of just appearing real.
Further reading: Turing test, Chinese room argument.