Bigger Models, More Data: The Key to Chinese AI's Potential

Published Date: 08/07/2024

China's AI landscape is evolving rapidly, with tech giants and start-ups alike investing heavily in large language models. But what's the secret to unlocking their full potential?

The future of artificial intelligence (AI) in China lies in the development of larger models and access to more data, according to Jiang Daxin, founder of Shanghai-based AI start-up Stepfun. Despite China's disadvantages in investment and advanced chip technology, Jiang believes that the 'scaling laws' of large language model (LLM) development will ultimately prevail.

Scaling laws refer to the relationship between AI model performance and their parameters. Essentially, they suggest that larger models, more data, and greater computational resources lead to improved performance, albeit with diminishing returns. Tech giants have been investing heavily in advanced technology, such as Nvidia chips, to gain a competitive edge.

Jiang's remarks were made at a side event for the World Artificial Intelligence Conference (WAIC) in Shanghai, where he noted that the upgrades of OpenAI's GPT series, which powers ChatGPT, and the substantial investments in supercomputing centers by tech giants like Amazon, Microsoft, and Meta, validate the effectiveness of scaling laws.

China's Big Tech firms and start-ups alike have been quick to launch their own LLMs since OpenAI launched ChatGPT in late 2022. There are now over 200 AI models in the country, including Tongyi Qianwen by Alibaba Group Holding and Ernie by internet search giant Baidu.

However, few Chinese AI firms have been able to match US tech giants in LLM spending, with many opting to develop client-facing applications that can generate revenue. Stepfun, founded in April 2023, has focused on developing fundamental models, including the recently launched Step-2, a trillion-parameter LLM, along with the Step-1.5V multimodal model and the Step-1X image generation model.

Apart from scaling laws, Jiang emphasized the importance of multimodality in constructing a world model. World models allow for the development of internal representations of external environments by processing visual and other types of data. Stepfun aims to unify generative and comprehension capabilities within a single model.

The company also operates consumer-facing products, including a ChatGPT-like personal assistant named Yuewen and Maopaoya, an AI companion that takes on the personalities of specific characters.

According to Alex Zhou Zhifeng, managing partner at Qiming Venture Partners, global AI investments reached $22.4 billion last year, with 70-80% focused on companies developing large models. Zhou predicts that investments will shift towards AI applications, which are benefiting from decreasing token costs.

Peng Wensheng, an economist with China International Capital, expects the size of China's market for AI models to reach approximately 5.2 trillion yuan (US$715.1 billion) by 2030. The industrial AI market size is expected to be around 9.4 trillion yuan.

  Information 

Stepfun is a Shanghai-based AI start-up founded in April 2023 by Jiang Daxin, a Microsoft veteran. The company focuses on developing fundamental models and operates consumer-facing products.  Qiming Venture Partners is a venture capital firm that invests in AI and technology companies. China International Capital is a leading investment bank in China.

FAQS:

Q: What is the main challenge facing Chinese AI firms?

A: Chinese AI firms face challenges in terms of investment and access to advanced chip technology.

Q: What is the concept of scaling laws in AI development?

A: Scaling laws refer to the relationship between AI model performance and their parameters, suggesting that larger models, more data, and greater computational resources lead to improved performance.

Q: How many AI models are currently in development in China?

A: There are over 200 AI models in development in China, including Tongyi Qianwen by Alibaba Group Holding and Ernie by internet search giant Baidu.

Q: What is the focus of Stepfun's AI development?

A: Stepfun focuses on developing fundamental models, including large language models and multimodal models.

Q: What is the expected size of China's AI market by 2030?

A: The size of China's market for AI models is expected to reach approximately 5.2 trillion yuan (US$715.1 billion) by 2030, with the industrial AI market size expected to be around 9.4 trillion yuan.

More Topics :