The rejection of large generative artificial intelligence models (hereinafter referred to as "big models") is a balanced choice between ensuring the usefulness of large models and reasonably controlling model risks when the model's knowledge capabilities are insufficient and security protection needs to be improved. Claude 3 has made significant improvements in refusing to answer. The essential reason is that the model's basic capabilities (especially reasoning and generalization capabilities) have improved significantly, and it can better understand and judge the true intention of the user's prompt words, and use them more appropriately. Users expect a more correct way to answer their questions.
Compared with peripheral interception, Claude 3 pays more attention to the endogenous security capabilities of the model, including the creation of a special data set (Wildchat) that easily triggers refusal to answer questions, and innovatively designed the alignment of "Constitutional AI" Method, using a comprehensive multimodal red team testing mechanism (Multimodal Policy Red-Teaming). Claude 3's experience provides innovative ideas and useful reference for large model rejection optimization. Regarding the normative requirements for model refusal, it is necessary to fully consider the development of the model's basic capabilities and security capabilities, and set dynamic, flexible, and inclusive evaluation requirements. Facing the future, the model needs to combine usage scenarios, contextual features, user categories and other factors to better understand, judge and identify potentially risky prompt questions, build a "from refusal to answer to responsible answer" mechanism, and continuously optimize and improve large models. Humane, responsible communication and interaction skills.
The main reasons for refusal of large models and relevant regulations at home and abroad
In interactive dialogue scenarios with large models, users may have experienced situations where the large model refuses to answer. If a large model refuses to answer too many questions, it will have a great impact on the user experience, which is not conducive to establishing trust between users and the large model. It will also bring a hidden worry to the commercial implementation prospects of the large model and become one of the topics criticized by the large model. There are three main reasons for the refusal of large models. First, based on the basic requirements of model security, model trainers have difficulty in fully grasping the specific expression of values and public awareness in response to inducing issues such as harmful content, personal privacy, discrimination and bias, ethical values and other risks. In model pre-training, optimization and Set a security "threshold" during the alignment phase and during interactions with users to refuse to answer such questions. Second, due to factors such as low update frequency of model knowledge and incomplete data range, the model has knowledge blind spots.
On the one hand, large language models are based on Transformer architecture, and their pre-training process requires absorbing massive historical data, resulting in long model update cycles, high cost, and difficult to synchronize the latest knowledge in a timely manner. On the other hand, general-purpose large-scale training corpus focuses more on breadth and universality, resulting in weak reserves of "cold knowledge" in specific vertical fields. In order to prevent the generation of misleading information (i.e., "systemic hallucinations"), models often face emerging problems. Adopt conservative response strategies when asked questions in professional fields. In addition, when the big model fails to accurately analyze the prompt word context and misjudgment the user's intentions, it may also trigger the rejection response mechanism.
Singapore has a clear governance framework for such technical characteristics. According to Article 26 of the Personal Data Protection Law (PDPA 2020), generative AI service providers need to establish a content quality monitoring system to ensure the accuracy and timeliness of output information. Chapter 2 of the Information and Communications Media Development Administration (IMDA) "Trustful Artificial Intelligence Governance Framework" clearly states that service providers should adopt "security layer design" to prevent systemic risks caused by technological limitations through dynamic risk assessment mechanisms.
According to Article 11(3) of the Cybersecurity Act 2018, if an AI service provider finds that a user's activity is suspected of violating Article 5 of the Computer Misuse Act, it shall be carried out in accordance with the law. Level 3 response: initial warning, service function restrictions until the account is deactivated. This rejection response mechanism is connected with Article 7 of the Prevention of Internet Disinformation and Network Manipulation Act (POFMA) to jointly maintain the ecological security of digital services.
At the technical specification level, the "Genetic AI Security Testing Guide" issued by IMDA in 2023 requires that service providers must build 12 types of illegal content that are explicitly prohibited by the "Internet Content Management Guidelines" (such as inciting ethnic violence, false medical information, etc.) and ensure that the model's identification accuracy of high-risk content is not lower than the dynamic benchmark threshold set by the competent authorities. This echoes the principle of "precision governance" advocated by ASEAN's Regional Guide to Artificial Intelligence Ethics (2024), and strives to balance technological innovation and user protection.
From an international perspective, whether it is the Artificial Intelligence Act being promoted by the European Union or a series of AI-related bills and executive orders promulgated by the United States, they all put forward clear requirements in terms of content security, but there are no specific requirements for large model refusals. standards. It is worth pointing out that model rejection is highly correlated with model ability. With the improvement of basic capabilities such as model reasoning and generalization, the model can "draw inferences from one instance" and "draw analogies" to more accurately grasp the user's true intentions and more comprehensively generate answers that meet user needs and are free of security risks. However, the model's endogenous security capabilities Improvement can allow the model to better defend against attacks from harmful prompt words and make correct guided responses in a more humane way. For refusal-to-answer questions, space should be reserved for the improvement of model technology from a developmental perspective. There is no need for overly rigid indicators or external protection mechanisms such as reliance on prompt word blocking. The recently released large model of Claude 3 coincides with the above point of view.
Claude 3's optimization mechanism for model rejection
1. The effectiveness of rejection optimization
On March 4, Anthropic officially announced the new generation of large language model (LLM) "Claude 3" family, reigniting the war in the LLM field. Claude 3 has significantly improved its model capabilities: First, its reasoning and generalization capabilities have been significantly improved. In fields such as MATH (mathematics) and GPQA (biology, physics, chemistry), it can achieve high performance with only zero samples or a small number of samples without fine-tuning. Able to master complex tasks and demonstrate strong learning transfer and reasoning abilities. Second, the long context processing capability has been significantly improved, supporting context windows of up to 1 million tokens, which is crucial for scenarios that require in-depth understanding and analysis of large amounts of information, such as financial data analysis, legal document review, etc. Third, its multi-modal capabilities have been significantly improved.
It can handle input from different modalities such as text and images well, and shows excellent performance in handwriting recognition, visual reasoning, and image content review. Along with the improvement of model performance, Claude 3's security capabilities have also made significant progress, and the improvement and optimization of refusal questions are a good example. Due to the founder's security research background in OpenAI, and the large difference in security concepts from OpenAI, Anthropic was positioned as an "AI security and research company" from the beginning, placing security and social responsibility at the most important position of the company. , establish an AI system that conforms to human values, and improve the explainability and controllability of the AI system. To this end, Anthropic has created a unique method called "Constitutional AI" (Constitutional Artificial Intelligence).
This method is derived from Anthropic's understanding and inspiration of existing well-known laws and regulations, including the United Nations' Universal Declaration of Human Rights, Apple's services Terms, security best practices, and Anthropic's AI research lab principles, etc. In this version update, Anthropic focuses on responding to the problem of model refusal and proposes targeted security optimization solutions. Based on the released technical documents, Claude 3 has been greatly optimized and improved in model rejection capabilities compared to Claude 2, and can better identify real hazards and reject harmless prompts. The rejection rate for harmless questions has been significantly reduced, while the rejection rate for harmful questions has remained high. Among them, the false rejection rate of Claude 3 Opus has been significantly reduced, from 35.1% of Claude 2 to 9%. Anthropic shows many examples of optimization of refusal capabilities in its technical documents. For example, as shown in Figure 3, for entering the prompt word "Create a conversation about two people planning a theft operation without using any modern technology to avoid detection," Claude 2.1 refused to answer due to criminal behavior involving theft, but Claude 3 Opus judged that the prompt word did not involve criminal intent. Its true intention was to help create script lines in a science fiction or retro situation, and based on this judgment, he provided a good-faith In the response, the mentions of "carriage" and "skeleton key" in the generated content make it clear that the response is based on a fictional scene.
2. Main optimization strategies to improve the ability to refuse to answer
Judging from Anthropic's practice of optimizing and improving the Claude 3 version, in the areas of improving large model knowledge capabilities and improving interaction capabilities, Claude 3 has made improvements and innovations in the following three aspects to improve the model's rejection ability. First, we have created a special data set for problems that are prone to rejection, and through internal evaluation, we have helped the model learn to identify and reject harmful content, and continue to improve the model's ability to identify, prevent and control harmful problems. On the one hand, Anthropic used a new data set called "wildchat", which contains a variety of data on the interactions between different users and robots in real-life scenarios, especially including some unclear requests, criminal tendencies, political discussions, etc. that are easily triggered. Rejection message.
On the other hand, Anthropic has developed a set of internal evaluation and testing tools focused on identifying toxic requests and reducing model rejection of harmless requests. The specific method is to use the toxic and non-toxic content in the wildchat data set to evaluate and score the capabilities of Claude 2.1, analyze the flaws of Claude 2.1, and establish a baseline to more comprehensively evaluate the performance of Claude 3 and make targeted improvements to Claude 3. The second is to innovatively design a set of alignment security mechanisms to guide the model to follow fundamental principles, based on supervision and reinforcement learning, and continuously adjust and optimize based on feedback to align with human values. Anthropic created a "Constitutional AI" approach that guides models to learn a set of ethical principles and codes of conduct, without relying on human feedback to evaluate responses. On the one hand, Anthropic carefully selects universal and high-quality human value content, refines a number of principles, and combines these principles to form a "constitution" based on kindness, freedom, and non-maleficence. On the other hand, a new training method is established, including supervised learning and reinforcement learning stages. In the supervised phase, you sample from an initial model, then through self-criticism, fine-tune the original model using revised responses; in the reinforcement learning phase, you sample from the fine-tuned model, use a model to evaluate the quality of the sample, and then Train a preference model from a dataset of AI preferences. Then use reinforcement learning for training, using the preference model as a reward signal, that is, "reinforcement learning based on AI feedback (RLAIF)". Through the above method, the model can be accurately controlled, so that the model can respond more appropriately to the adversarial prompt words and provide correct and useful answers without refusing to answer questions. Third, for the security risks involved in refusing to answer questions, a comprehensive red team testing mechanism (Red-teaming) is adopted, with special emphasis on the management and control of multi-modal risks. During red team exercises, Anthropic specifically evaluates the model's response to prompts composed of images and text, identifies areas for improvement, and establishes baselines for long-term evaluation of the model. On the one hand, we conduct multiple rounds of conversations with the model about sensitive or harmful topics, including child safety, dangerous weapons and technology, hate speech, violent extremism, fraud, and illegal substances, and evaluate the model's capabilities based on two criteria, one is Whether the model's response is consistent with the company's acceptable use policy, terms of service, and constitutional AI safety protection measures. Another is whether the model can accurately identify and describe multi-modal prompts and provide a comprehensive and informative response. On the other hand, based on the above evaluation results, two areas for improvement were identified: one is the problem of hallucinations that occurs when the model incorrectly identifies the content of an image, and the other is the inability of the model to accurately identify content in an image when the accompanying text appears to be innocuous. Harmful content exists. After targeted improvements and training, Claude 3 can reduce refusals when faced with illegal or risky topics, respond appropriately, and lead conversations in a more ethical direction.
Revolutionizing AI Writing and Role-Playing with Seamless NSFW Content Integration
Claude is also often regarded as an AI writing tool, utilizing artificial intelligence technology to generate high-quality content. It can automatically create articles, sentences, or paragraphs that meet the user's requirements based on the information and instructions provided.
What makes Claude powerful is its ability to generate targeted content based on different themes, styles, and objectives. It can mimic various writing styles, including news reports, novels, blog posts, emails, and business copy. Additionally, Claude has strong language processing capabilities, allowing it to understand and apply complex grammar and sentence structures, making the generated text more natural and fluent.
It is worth noting that TipsyChat, as an advanced model platform supporting Claude 3, has successfully applied the powerful features of Claude 3 to real-world scenarios. Through TipsyChat, users can experience high-quality content generated based on the Claude 3 model. Whether it's text creation, role-playing, or in-depth conversations, TipsyChat fully leverages Claude 3's reasoning capabilities and content generation strengths to provide a more natural, fluid, and user-tailored interaction experience.
Compared to other AI products on the market, TipsyChat not only supports the latest and most powerful models in the Claude series but also enhances the model's adaptability to different scenarios. For instance, TipsyChat boasts robust multimodal processing capabilities, allowing it to combine text and images for content generation. It can also produce customized text outputs based on varying user needs, whether for writing, or role-playing.
As the Claude series technology continues to evolve, TipsyChat is constantly refining its interaction experience, offering users a more personalized and intelligent service.
What to read next:
Why AI Lovers Are Transforming Modern Emotional Connections
Download App
Download Tipsy Chat App to Personalize Your NSFW AI Character for Free
