ChatGPT application and thinking

1. ChatGPT application

ChatGPT is of great significance to the application of AIGC in text mode. It can be attached to dialogue products and carriers, including but not limited to content creation, customer service robots, virtual humans, machine translation, games, social networking, education, and family care. and other fields. These may be the directions where ChatGPT can quickly land.

Some of these directions will involve a comprehensive reform of interaction. For example, machine translation is no longer the traditional text input -> real-time translation, but will appear in the form of assistant question and answer at any time. Even give a roughly general Chinese meaning, let the machine give the corresponding English. The writing products we are currently making may also involve changes and innovations in the creative model.

Some directions will improve product quality in an all-round way, such as existing customer service robots and virtual humans.

ChatGPT, as the basic model of text morphology, can naturally be combined with other multimodal

For example, the Stable Diffusion model, which is also popular recently, uses ChatGPT to generate a better prompt, which provides a strong power of text form for AIGC content and increasingly popular artistic creation.

ChatGPT’s replacement for search engines: Chat GPT can be used as an effective supplement to search engines, but as for whether it can replace search engines (a place that many people pay attention to), regardless of the reasoning cost, it is still too early to talk about the effect. .

For queries that have an answer on the Internet, extraction is completely sufficient, and now friends have such a function recently. There is no clear answer on the Internet, and even if relevant materials are retrieved (ChatGPT should not have such a function), no one can guarantee the credibility of the generated results.

Upgrade of ChatGPT itself

The combination with WebGPT updates the information in real time and judges whether the facts are true or false. The current ChatGPT does not have real-time update and fact judgment capabilities, and if this is combined with WebGPT's automatic search capabilities, ChatGPT can learn to explore and learn from the massive knowledge base by itself, and prediction may be a capability of GPT-4.

There are many more directions, including the combination of ChatGPT and recent mathematical logic work. Due to the limitation of personal thinking, it is impossible to list them one by one.

2. Thoughts on ChatGPT

In the past 2 years of articles related to the OpenAI GPT language model, the RLHF method has a remarkable effect. The core of ChatGPT's success lies in the RLHF (Reinforcement Learning from Human Feedback) based on LLM (Large language model). It can be said that RLHF is a promising and interesting direction; reinforcement learning has a high probability of playing this key role in the upcoming GPT-4.

Combined with our views on ChatGPT, we elaborated from the perspective of algorithm and industry updates:

First of all, there is currently no more information to support the scale of ChatGPT, so it is impossible to clarify at what scale such a smart ChatGPT was achieved.

The original 175B had the GPT-3 designation Davinci, other size models have different designations. However, the code name since then is almost a fog, not only does it not have any papers, nor does the official introductory blog. OpenAI claims that Davinci-text-002/003 is GPT-3.5, and they are all InstrucGPT type models. ChatGPT is based on one of the fine-tuning models, so it is speculated that ChatGPT may be a 100 billion model.

Secondly, ChatGPT is not exactly a breakthrough innovation. It is an almost natural result of OpenAI’s step-by-step solid work accumulation, and it belongs to the summary of the industry’s development in the past two years.

You generally don’t have the opportunity to touch the 100-billion model (there were few open-source 100-billion models before, and GPT-3 was also charged), and you don’t know the capabilities of the current 100-billion model, and you can’t estimate the full fine-tuning of this level of model. The early Transformers represented by Bert and T5 are not in the same order of magnitude as the current large model. In fact, a new text-davinci-003 was added to OpenAI on November 28, but it hardly caused any domestic discussion. If ChatGPT (released on 11-30) was not a free trial, it might not have caused such a big response.

The work in the same period also included Deepmind's Sparrow and Google's LaMDA, and the effect should be comparable to ChatGPT. Similarly, the above-mentioned WebGPT and Cicero do not have much splash in China. In the past two years, the development of LLM has reached this level. Perhaps because of the cost or the difficulty of engineering, it has been neglected in China at some level. And this time ChatGPT just found a good "exposure point" and became an instant hit.

Therefore, on the one hand, we have to rationally view the achievements of ChatGPT, but on the other hand, the emergence of ChatGPT will bring our understanding and foreign advanced ideas into line. We should think about how to use these exciting latest achievements, and among them The key is how to find the way that suits our entrance.

Third, data processing is not simply labeling, and excellent data is also a great advantage. Apart from technical considerations, OpenAI rarely open source data, obviously they have also put a lot of effort into the data, the quality of training corpus and open source C4 or The Pile can not be mentioned in the same breath.

Of course, the open source billion-dollar model we currently use at our core has a lot of capabilities to be tapped. Due to its lack of generative dialogue and question answering in fine-tuning tasks, some performance is not as expected as ChatGPT. But for many tasks, with In-context Learning, this gap will be further narrowed.

3. How to learn from and use ChatGPT

For the reference and use of ChatGPT, it can be roughly classified into the following four directions:

Direct use layer This level is an excellent part of the reuse API, and the advantage of direct use is that it can quickly realize multi-granularity and multi-level functional requirements. In many cases where the requirements are difficult to define clearly and the data is difficult to obtain, it is extremely profitable to reuse and package such functions.

Of course, its shortcomings are also obvious. The cost of direct calling is extremely high. According to the cost estimation of GPT3.5 (Davinci): 1k tokens ≈ 700 words is 0.02 US dollars. After conversion, an article with 2k words needs 0.4 RMB to call directly. If it is conservatively calculated based on 10,000 daily active users and 10 articles per capita, the daily call cost is: 10000*10*0.4=40000 yuan. Although the cost is too high, the implementation time is the least.

In addition, according to the conversation with the OpenAI staff on Musk Twitter, it can also be seen that each chat process costs a few cents, so the direct call of ChatGPT is extremely costly.

Indirect usage level

The core idea at this level is to use the OpenAI interface to generate high-quality data according to different needs, and overcome the bottleneck of existing data; and then use the existing open source large model for data amplification. This is currently more practical and takes less time to implement. A compromise between time cost and effect.

Ideas for reference

First of all, the group currently has preliminary attempts to refer to RLHF methods, such as labeling multiple candidates, using the obtained labeling results to re-fine-tune the generation model, or adding RL learning to the ranking stage. Secondly, we also try some efficient parameter tuning methods to fine-tune the existing large models. However, this article is subject to resources that need to be evaluated and confirmed.

In general, the rewriting will be extended from the initial seq2seq to the GPT+Instruction Tuning path.

Interactive upgrade

Make the writing as a whole in the form of ChatBot. This core idea is introduced in another report on the dialogue system, which involves changes in the interaction level. But the emergence of ChatGPT and core technology make it possible to upgrade the form. With the development of deep learning and multi-agent systems, there will be various, diverse and multi-functional X-Bots in the future.