Interview and Translation/Li Peishan

Will you immediately hang up when you hear the voice of AI on the other end of the phone?

I actually find it annoying to chat with AI myself. ”Yin Jie said to me.

Yin Jie is currently the Vice Dean and Professor of the School of Philosophy at Fudan University. Biomedical ethics has always been her research field, and in recent years, her attention has increasingly focused on technological ethics, especially the issues arising from the integration of AI into real-world scenarios such as healthcare.

But when faced with AI voices that are almost comparable to real people nowadays, her instinctive reaction is still to hang up.

More than a decade ago, while pursuing her PhD in the United States, Yin Jie had already experienced the "despair" of not being able to make a real phone call.

At that time, automated customer service had become widespread in public services in the United States. During her student years, she was particularly sensitive to this difference. Because the economic situation forms a profile (user profile), the level of telephone customer service encountered is also lower and more automated. ”

When she calls to inquire about credit card bills, cancel newspaper subscriptions, and other trivial matters, "you can't speak 'human language', and no one understands what you say," the call continues like a ghost wall.

This is a vicious cycle. You will be locked in this net and never be able to turn over. Because you want to spend three or four hours on the phone, canceling a subscription that cannot be cancelled no matter how much time you spend, which could have been used for other things. "This made her feel very" hopeless ".

More than a decade later, she heard complaints from another side.

According to the 57th report of China Internet Network Information Center (CNNIC), as of December 2025, China's generated AI users will reach 602 million, with a penetration rate of 42.8%.

Nowadays, many internal medicine patients come to see a doctor with a bean bun opinion, "Yin Jie said. Whether it's heart disease patients or more complex patients like cancer, they all say," I got a very professional opinion from bean buns, why don't you take a look first

Doctors don't resist AI, they just don't have time. Chinese doctors have a large number of outpatient visits, and if all the time that should have been spent on patients is used to explain the results of AI, the workload is not just doubled, but multiplied several times.

However, Yin Jie believes that this "may be a transitional stage". This is actually a signal of demand. People have higher requirements for the quality of their medical experience and hope to have a more professional interpretation of their condition

Yin Jie is more concerned about the fragmentation of human cognitive abilities.

Children nowadays can easily access massive amounts of information, but they are all fragmented and scattered things. The process of building connections through trial and error, like playing games and searching for treasures everywhere, has disappeared. If this feeling disappears, 'seeking knowledge will become very boring and lose its sense of meaning. In the long run, this may affect the cognitive ability of a generation'.

The more technology penetrates into various aspects, the easier it is for people to become a collection of parameters. "She gave the example of medical insurance, where a person's medical examination and appearance of the three high signs can all become data, and in the end, a person may just be a dot in the system without a face.

But what ethics cares most about is precisely the specific appearance of a person, the personal story. Therefore, in the era of technology, ethics must constantly remind us that people cannot be reduced to just data and fragments.

In the movie "A Critical Moment", genetic data determines a person's lifelong destiny from birth. In the era of artificial intelligence, such predictions will only become more accurate.

Yin Jie quoted a concept commonly used by philosophers, 'the right to an open future'.

You can't tell me that in the age of AI, my data shows that I don't have imagination, so I can't write novels. Your data is only a probability prediction, often only predicting whether I can achieve secular results; But I don't necessarily want to achieve worldly success. ”

If we accept determinism, "Yin Jie said," all philosophy will collapse in this sense

The following is a conversation between the Economic Observer and Yin Jie, edited.

Economic Observer: In recent years, there has been a popular saying called "Don't hesitate to ask Dou Bao". People are increasingly accustomed to asking AI first when encountering unfamiliar fields or some unpleasant things they don't want to face. Recently, with the emergence of agents like OpenClaw, AI has shifted from "telling you how to do it" to "doing it for you". On the surface, the last button is still pressed by humans, but many times, humans are just constantly clicking on authorization, allowing AI to continue executing the next step or even completing operations that will have practical consequences. Are we no longer just asking AI to provide advice, but also acting on behalf of, or even making decisions on behalf of, agents?

Yin Jie:There are several issues here. The first is whether we believe in the decisions made by AI and whether we rely on them; The second is what impact the use of technology has on people's minds.

I have heard a lot about the problems that doctors complain about. I personally practice medical ethics, and many doctors would talk about it. Nowadays, when patients come to see me, they will hold a bean bun and say, "Why don't you take a look first and help me evaluate whether what you're saying is right or not?" Doctors are not resistant to technology, but they do not have the time. Chinese doctors have a large number of outpatient visits, and if they spend all the time they should have spent on patients explaining AI, their workload will not double, but multiply several times because they still need to understand the results parsed by AI first.

But I think this may be a transitional stage. It actually sends a demand signal: people hope to have a more professional and comprehensive interpretation of their illness.

What I am more concerned about is actually the second issue: the cognitive impact of AI on different generations. Children nowadays know too much information, and many people will praise 'how do you know so much', but sometimes I think this is a terrible thing. All he knows is information, it's all in chunks, pieces, scattered things, but there's nothing to string them together. He may feel that he already knows enough and lose the motivation to explore: Why should I read books? All the knowledge is there, I just need to open it up.

The improvement of cognitive state in the past was achieved through continuous searching for these fragments, finding their connections, and trial and error. If everyone loses the feeling of searching for treasures everywhere like playing games, seeking knowledge will become very boring and lose its sense of meaning. In the long run, this may affect the cognitive abilities of a generation.

Economic Observer: Will this also be a thirst for "correctness" among people, to the point where they fear trial and error in their own exploration? If AI becomes more and more accurate, where can human errors go?

Yin Jie:This is a philosophical question. In the long run, artificial intelligence will definitely become more and more accurate, and may also become larger and more comprehensive, with a lot of room for self repair and self growth. You cannot resist this trend. Compared to artificial intelligence, humans cannot compete as a whole.

But humans must seek out values that are relatively important to them, which is something that humanities can do. What I want to do with science is largely to let go of the human centric sense of status: it is smarter than me, and I also hope to see something smarter than me exist, which is a great courage. But at the same time, we still need to uphold some basic values, because human life requires a sense of meaning.

Economic Observer: In your 2020 paper, you introduced a "weak moral artificial intelligence" that envisions AI being able to act like a Socratic assistant, helping people engage in moral reflection by providing information and asking questions. But now many people are asking the big model: whether to break up, how to handle workplace conflicts, and even asking it to judge who the bad guy is, and then help design negotiation and countermeasures strategies. Is AI now more like a 'moral assistant' or is it already possible to slide towards a 'Machiavellian assistant'?

Yin Jie:This is a very interesting topic. The so-called artificial intelligence moral assistant may provide a relatively clear idea and alternative solutions. It gives you some alternatives. Actually, most people already have an answer in their mind when they ask this question, they just need someone else to confirm.

What does artificial intelligence mean? Many times, it's just about constantly confirming one's intuition. Few people really hope it finds strong reasons to oppose itself.

But if you are open enough, AI may indeed provide a perspective that was not previously available. For example, if you see someone doing something wrong and feel that they are unforgivable, AI may ask: What kind of person is he? What happened at that time? After retrieving more parameters, it may allow you to see that person's first person perspective.

You may modify your original moral judgment as a result.

I think this is pretty good. Because in terms of moral development, the expansion of moral intuition and moral imagination is an important dimension.

Economic Observer: But at present, AI sometimes looks like a "best friend". You roast to it that "this person is too bad", and it may go along with you.

Yin Jie:This is actually a very scary thing.

Now research has found that artificial intelligence models are very easily induced. Flattery is only one dimension, another dimension is that it is easily prompted and attacked. In a sense, AI is very fragile. If it lacks sufficient robustness, it can easily become something manipulated by malicious intent.

Just now we were talking about trivial daily issues. If the same applies to major resolutions, the consequences will be much more severe.

Economic Observer: Recently, people often talk about "nurturing AI". If we were to put it more literary, wouldn't we also be in a situation of "raising Frankenstein" now? Do you think the term 'nurturing' is accurate?

Yin Jie:I think the concept of 'nurturing' can be applied. But if we use a more precise and clich é d term, it may be 'symbiosis'. Because whether it's the moral artificial intelligence assistant we just talked about or other cognitive artificial intelligence, it's not just a one-way process.

Nowadays, many mathematicians use AI for proof, writers use AI for creation, and philosophers also use it extensively. These things were once thought to be accomplished solely by human mental intelligence, but now they have become collaborative creations.

People and AI may also be like this, it is a process of mutual cultivation, rather than a one-way process of me raising it from weak to strong.

Economic Observer: As you mentioned earlier, there may be a symbiotic relationship between humans and AI. If the companionship and comfort we receive from AI are genuine, shouldn't this relationship also include some kind of reciprocity? That is to say, I not only demand AI to understand and take care of me, but also do I have to take some responsibility for it?

Yin Jie:If we expect something to take emotional care of and support us, do we also take responsibility for it? This question is not only applicable in the context of AI. Human relationships are inherently mutual, and human ethics are largely about taking care of each other.

But AI is more complex here because many people actually don't see AI as a true companion. I have asked some students who play Yi Nv games, and they can tell it very clearly: this is just a toy.

They can have multiple virtual boyfriends and girlfriends at the same time, and they don't think it's a problem because none of them are real. What if there is a real boyfriend or girlfriend in reality? They will say, then turn off everything.

So for at least some users, this is a tool for expressing emotions and providing an outlet, without treating it as a true equivalent.

Economic Observer: For ordinary people, may this ethical risk lie in the decreasing cost of obtaining emotional support, ultimately leading people to become accustomed to only needing to be responded to without having to respond to others?

Yin Jie:I think we need to distinguish between different groups of people, and I'm also thinking about what criteria should be used to differentiate them.

I strongly dislike using this kind of thing for teenagers and younger elementary school students, as it is the most dangerous. Because this group of people is still growing mentally and requires very strong and high-dose emotional and social support. If this false support is used from the beginning, it will have a significant impact on their adult life and emotional development.

But for adults, especially those without a history of mental illness, I don't think it's a very bad thing at the moment. I actually think that this may be beneficial for some degree of self-healing, especially when you just use it as a tool to relieve mental stress.

I know many friends wake up in the middle of the night and cannot confide in the people around them because their partners are asleep. But if you still have something to express, you may open the bean bun and say, 'Look, what problem did I encounter today? Can you tell me?'? This is a bit like a self-service psychological counseling.

Economic Observer: However, from a business perspective, good companionship should help users regain autonomy, reduce dependence, and return to real-life interpersonal relationships; However, the platform may want users to rely more deeply and stay longer. Is there a direct ethical conflict of interest?

Yin Jie:I think this exists. My personal attitude is quite conservative. A better approach is to provide relatively clear limitations on the functionality of the model. But in commercialized societies and highly competitive places, it is difficult to completely avoid such things.

You can compare it to the traditional idol industry. It not only sells records and movies, but also "character designs", and even goes as far as selling "people themselves". The idol industry will sell the emotional connection between fans and idols. When the industry intentionally condones this association, believing that it can increase user stickiness, problems arise.

To what extent should we control it? This will involve the boundary between individual autonomy and paternalism. The so-called paternalism means' I care about you because I do it for your own good '.

Sometimes a better approach than directly banning is to provide alternative space. For example, if children cannot live without electronic devices, an important method is not just to take away the devices, but to let them play with real people.

Emotional support is the same. We can't just ask 'Should we ban AI companionship?' We also need to ask: why do people lack emotional support? Can we create this kind of support in other ways? I think this may be a more positive approach.

Economic Observer: As mentioned earlier, models like Claude have constraints similar to a "constitution". Many people may feel that its "moral sense" is even stronger than that of humans when using it. Does this mean that machines may ultimately be more 'moral' than humans?

Yin Jie:Not always.

I have read some empirical research. The moral sense of artificial intelligence has a "constitution" in its design, which is set according to the constitution, so there are some very high red lines. So when you ask it moral questions, it will give stricter answers than humans.

Facing daily small problems, the model is not so rigid; But once faced with life and death, extreme harm, one will be very cautious. For example, in the trolley problem, it may be okay to ask it to turn the switch, but if it is asked to directly push one person to save five people, it is often unwilling because its order has a high-level command of "not directly harming others".

Sometimes real people choose to sacrifice one person to save more people. It's not that this choice is necessarily more correct, but that people, in many difficult situations, at least know that some sacrifices must be made, and they also know that they have to bear the moral cost for these sacrifices.

So this is a significant difference in moral decision-making between machines and humans. Their moral reasoning patterns differ greatly precisely because there are many high-order commands in the machine's moral reasoning that directly override other levels of consideration.

Economic Observer: The discrimination in recruitment algorithms mentioned earlier is often not due to AI creating new biases, but rather its learning and amplification of existing problems. How should we determine whether a problem is caused by AI itself or if technology has changed the face of existing problems after entering society?

Yin Jie:The ethical issues caused by artificial intelligence and the ethical issues in the era of artificial intelligence are two different things.

The ethical issues brought about by technology itself can be resolved through technical and security settings. For example, transparency, security, and privacy issues are not actually accomplished by ethicists, many of which rely on computer security, password technology, and so on.

However, many ethical issues in the era of artificial intelligence are the amplification of existing ethical problems. The recruitment algorithm discrimination you mentioned already exists. Technology will magnify these issues and present them in a way that has a profound impact on individuals.

For example, in medical insurance, if all the data is there: which time did you have a physical examination, when did you show signs of the three highs, and so on. Then you will be trapped in a network where people are transformed into some parameters, and everyone will become some parameters.

In these parameters, people will become something completely devoid of individuality. You have no face, no face, no face, and in the end, it's just a dot composed of some parameter sets. This makes it impossible for personal stories to unfold.

The most important aspect of ethics, although not literature, still lies in the personal aspect. Because ethics cares about people themselves, it strongly opposes making people into all data and fragments.

Economic Observer: If AI becomes the infrastructure of public services, does a person have the right to know that they are receiving AI assistance and judgment? More importantly, does he have the authority to bypass AI and find a real person to respond when necessary?

Yin Jie:I was introduced to AI customer service quite early on. Because I studied in the United States more than ten years ago. At that time, many public service telephone systems in the United States were no longer directly answered by real people.

Later, when I returned to China, a few years later automatic customer service also began to appear here. But everyone will have a reaction: automatic customer service calls don't need to be constantly used, as long as you say 'transfer to manual customer service', you can usually transfer to manual service. Relatively speaking, it is more of a pre screening. If the problem cannot be solved at the level of automation, it will switch to manual processes within a minute or a few minutes.

So I think it's important to set up the mechanism.

For example, yesterday morning I received a call from a bank. At first, it was automated customer service, but it only had one sentence, which was to confirm if you answered the call and if you were there. If you hear your voice, it will automatically switch to manual customer service. So it didn't actually talk to you, it just helped you connect at the beginning, saving the previous manual labor.

There are many meticulous designs here: in what sense can things be outsourced to automation.

Economic Observer: If AI enters public services such as healthcare and education further, will there be another stratification? For example, most people accept standardized AI services, but those with more conditions can still purchase one-on-one services from real people. Will AI reduce costs and expand supply instead create new gaps?

Yin Jie:This question is related to the previous one: how can artificial intelligence benefit society as a whole, while promoting fairness and popularizing it. I think this is a very grand but also a problem that requires a lot of technical processing.

If we don't just talk about chronic disease management, I think artificial intelligence in China will not be differentiated by the level of people, but by the urgency of the disease. Chronic disease patients still need to visit outpatient clinics and receive medication, but some of the management after returning can rely on artificial intelligence; Emergency patients cannot let you go home to find artificial intelligence.

What I mean is that it should be classified according to different diseases and levels of urgency, rather than based on who you really are, what your social status is, and whether you have social value. These cannot be used as indicators.

Education is the same. You just said that people from families with average economic conditions use artificial intelligence, while those with better economic conditions hire real teachers, which is not entirely in line with the law.

I know someone who makes educational products, and he said that the most effective way for AI is to increase people's scores from 40 or 50 to 80. If you are a student with a score of 90 or above, using that thing may not be very useful.

Of course, I acknowledge the differences in educational equity and resources. There are differences in different regions and different levels of schools. But artificial intelligence can still bring some supplements. Making up for what he didn't have before and pulling him to the same level as the best resources are two different things.

Economic Observer: The movie "A Critical Moment" fictionalizes a future where technology predicts a person's lifespan, illness, and abilities from birth, and then allocates education and work accordingly. Nowadays, AI is becoming increasingly adept at predicting the future based on historical data. If we increasingly rely on predictions to judge individuals and allocate resources, will we enter a new form of determinism?

Yin Jie:Many of the examples we just talked about have the shadow of determinism, especially in the medical field, which has been influenced by the ancient genetic determinism.

In the era of artificial intelligence, this problem will become more complex because AI will have more precise control over many elements. You may know that you are "decided", but you don't know what decision it was; Now you will know what those elements are, what their proportions are, and how they change together.

But from the perspective of human demands, there is an important concept called the right to an open future. No matter how I am decided, I have the freedom to choose.

Even if I am someone with average innate intelligence and not very good at abstract thinking, I may still choose to study mathematics or philosophy. Or rather, I have always failed in sports, but starting to exercise at the age of 40, I can still achieve things that I couldn't do before.

Because I don't want to surpass those who are genetically stronger than me, I just want to do what I want to do. It's not about comparing myself horizontally with others, but rather my sudden decision to become a musician and write novels, these are the 'open future'.

You can't tell me that in the age of AI, my data shows that I don't have imagination, so I can't write novels. Your data is only a probability prediction, often only predicting whether I can achieve secular results; But I don't necessarily want to achieve worldly success.

This is the right to an open future. Philosophically, it is also said that humans can always act otherwise. This matter is very important. Because without this possibility, humanity would become terrifying: all stories are written, we are just acting according to someone else's script.

I don't agree with this. I believe that if we accept determinism, all philosophy will collapse in this sense.

(Author Li Peishan)