ChatGPT main features
ChatGPT is essentially a dialogue model that can answer everyday questions, engage in multiple rounds of small talk, admit wrong replies, challenge incorrect questions, and even reject inappropriate requests. After announcing the blog post and trial interface last week, ChatGPT quickly "exploded" the network with its amazing conversational capabilities.
1. Technical background of ChatGPT
ChatGPT currently does not release papers, but only releases introductory blog posts and trial APIs. Judging from the technical points and diagrams provided in the blog post, it is consistent with the core idea of InstructGPT published earlier this year. Its key capabilities come from three aspects: powerful base large model capability (InstructGPT), high-quality real data (clean and rich), and reinforcement learning (PPO algorithm). The above three elements of the success of ChatGPT.
2. Main features of ChatGPT
What impresses users most is that it has a powerful language understanding and generation system. Its dialogue ability, text generation ability, and understanding of expressions in different languages are all excellent. It uses dialogue as the carrier, can answer a variety of daily questions, and enhances the memory ability and length of the history of multiple rounds of dialogue. Secondly, compared with large models such as GPT3, ChatGPT answers more comprehensively, and can answer and explain from multiple perspectives. Compared with previous large models, knowledge is "mined" more fully. It can reduce the cost of human learning and save time, and can meet most of the daily needs of human beings, such as quickly rewriting target text for humans, continuing to write large-scale novels, and quickly locating code bugs.
It is worth mentioning that it has security mechanisms and debiasing capabilities. However, ChatGPT adds a filtering mechanism to these two points. It can respond negatively and “smoothly” to inappropriate questions and requests.
Of course, Chat GPT is not perfect, and its shortcomings are obvious. First of all, its simple logic problem errors still obviously exist, and its performance is not stable enough (but overall it is much better than GPT3). Especially when there is a conversation history, it is easy to be swayed by misleading users.
Second, Chat GPT sometimes gives answers that seem reasonable, but are not correct or even absurd. Part of the answers need to be judged by themselves to judge whether they are true or false, especially when the user is in an unknown state to consult the model, it is even more difficult to judge the authenticity. Chat GPT allows producers to increase misinformation at low cost, and this inherent shortcoming has had some real impact. The programming question-and-answer website StackOverflow announced that it will temporarily prohibit users from posting content generated from Chat GPT. The website mods stated that the number of seemingly reasonable but actually wrong replies is too large, which has exceeded the capacity of the website.
Additionally, it is less resistant to insecure prompts and suffers from over-guessing user intent. This is mainly reflected in the fact that when the user's intention to ask questions is not clear, Chat GPT will guess the user's intention. The ideal situation should be to ask the user to clarify; when the user's intention is not clear, there is a high probability of giving an inappropriate reply. A large number of user feedback, Chat GPT part of the reply is more nonsense, the sentence is fixed. It often overuses some common phrases and sentence patterns. This is related to the fact that users tend to choose longer replies when constructing training data.