A year ago, the world community was shocked by the GPT-3 model, and more recently, LaMDA and MUM, AI models from Google, were presented. They will make a real revolution in chatbots, as well as significantly increase the ability of search engines. As an AI enthusiast, constantly watching the new technology examples, I wish to share with you the review of new levels of AI.
If you wish to know more about the latest ML trends, check this AI blog, it’s really impressive.
A new level in the race for AI was shown by China, which recently introduced a unique new AI model, which was called Wu Dao 2.0. This model simply has no analogues, and the closest to it in terms of GPT-3 development, is simply not able to compete with it. It has a really wide potential and incredible prospects.The full potential of the model has yet to be mastered, but it can already be stated that this will be quite difficult to do, since it was trained on 1.75 trillion parameters, which is more than 10 times more than that of GPT-3.
Initially, the FastMoE system was developed, very similar to the Mixture of Experts (from Google). Due to the fact that the model functions on PyTorch, it can be trained both on simple GPUs and on modern supercomputers.
Key Features of Wu Dao 2.0
The model was trained in 2 languages at once (Chinese, English). During the training, more than 4.9 terabytes of various images and texts were used. This is almost 8 times higher than the GPT-3 training set (570Gb).
Unlike other similar systems, Wu Dao 2.0 is multimedia, that is, it performs a whole range of different tasks, namely:
- generates text,
- creates deep fakes,
- recognizes faces, and
- creates images.
She is able to write poems, essays, and various couplets. In addition, it can create quite realistic images based solely on the description and vice versa. The model also predicts the three-dimensional structure of various proteins, which will eventually lead to many discoveries in medicine and pharmacology.
The Wu Dao 2.0 model is not inferior, in some components it surpasses the rest in terms of modern levels (SOTA) in 9 key tasks that have been recognized by the AI society (ideal indicator: result). This is a really impressive result, but it can be considered incomplete, since so far the number of comparisons of Wu Dao 2.0 and SATA in these tests has not been carried out. Therefore, it is necessary to wait for these tests to confirm this fact, but confirmation should not take long to wait and it will become another indicator of the awesomeness and superiority of Wu Dao 2.0.
The Wu Dao 2.0 artificial intelligence model is quite easy to train. This fact was demonstrated by the developers of the model during its presentation. To perform new tasks, the model requires a minimum amount of new information.
It operates on the basis of 4 research projects that are closely related to each other, namely:
- Wen Yuan
A training model that is open source. It reduces language confusion, while increasing efficiency, which should increase the speed of task completion.
- Wen Lan
A multimodal system that understands a connotative language. Identifies minor correlations directly between text and image to improve the effectiveness of the final result. It is possible to replace the text and image encoders.
- Wen Hui
It uses a reverse query algorithm, so that the system has almost achieved a performance similar to that of a human. It is the only model to date that uses the Generative Language Model (GLM). For better results, vector-based fine-tuning (P-tuning) is used.
- Wen Su
It is a mixed expert model. It supports many different hardware. It increases the learning rate (approximately 47 times higher than when using PyTorch). It is thanks to this that the model gets increased efficiency and speed of learning. This model allows us to work with ultra-long molecular structures.
However, despite this, the disadvantage of the model is that it does not have common sense, as well as various cognitive abilities. It is not able to conduct open dialogues or visual reasoning (although the developers claim that the model has feelings and common sense).
Continuous training
Wu Dao 2.0 artificial Intelligence can learn new features regularly. The main advantage among other models is that when learning new information and learning to perform new tasks, it does not forget the old information and can perform previous tasks. This feature brings AI even closer to the human brain and opens up new levels of development and learning for it. It is also possible that Wu Dao 2.0 will be able to learn to program, which will open up new unique opportunities for the model.
Hua Zhibing (one of the creators of Wu Dao 2.0) stated, ” Wu Dao 2.0 has certain reasoning abilities, and is also capable of emotional interactions.”
Many were in mild shock when testing the GPT-3. They simply did not expect such a result, but given that Wu Dao 2.0 is significantly superior in terms of performance and characteristics of GPT-3, the effect when using it will be much higher. However, such tests have not yet been conducted, but they are planned to be conducted soon.
The model passed the famous Turing test during which it competed directly with people and showed quite good results, which significantly exceeded all previous indicators.
Conclusion
The main advantage of Wu Dao 2.0 is multitasking and multimodality. These functions are necessary for strong artificial intelligence, as they allow you to perform a number of potential tasks. This model has attracted the interest of many major USA companies (Microsoft, Google, SpaceX). In the near future, global companies will also pay attention to it.
This model actively brings us closer to the developed artificial intelligence, and also significantly exceeds all current models. It not only shows all its power, but also serves as an impetus for the development of new models that should surpass it. The era of AI is approaching at a high speed and soon it will be possible to create a model that will not be inferior to humans, and in some aspects even surpass them.
