Xiaomi is reportedly in the process of constructing a massive GPU cluster to significantly invest in artificial intelligence (AI) large language models (LLMs). According to a source cited by Jiemian News, Xiaomi's LLM team had already established 6,500 GPU resources at its inception.
Guancha sought confirmation from Xiaomi, but as of publication, there has been no comment from the company. An insider revealed that Xiaomi, which previously maintained a low profile regarding LLMs, has been steadily increasing its computational chip reserves over the past few months to provide more robust computing power for its own LLM development.
Jun Lei plays crucial role in accelerating LLM R&D
Currently, AI is becoming a focal area for smartphone manufacturers and tech companies, with computing clusters being essential for training LLMs. Various indicators suggest that under Jun Lei's leadership, Xiaomi is accelerating its R&D progress in LLMs.
During his 2023 annual speech, Lei stated that Xiaomi would fully embrace AI LLMs. In April 2023, Xiaomi's AI Lab also established a dedicated LLM team, appointing Jian Luan as the head of this team, reporting to Bin Wang, vice chairman of Xiaomi's Technical Committee and director of the AI Lab.
Luan was previously responsible for the speech generation team within the AI Lab and has held positions such as researcher at Toshiba (China) Research Institute, senior speech scientist at Microsoft (China) Engineering Institute, chief speech scientist, and head of the speech team for Microsoft's Xiaoice.
However, an individual close to Xiaomi stated that the company is cautious about the substantial financial investment required for pre-training LLMs. They noted that lightweight models can have certain advantages over LLMs with hundreds of billions of parameters in specific tasks. Consequently, Xiaomi's focus on LLMs emphasizes "lightweight" and "local deployment."
Compared to other tech giants, Xiaomi boasts an extensive ecosystem encompassing smartphones, automobiles, and IoT devices. This breadth provides a competitive advantage as the AI LLM sector experiences intense competition while seeking practical applications. However, it also requires outstanding performance in the AI LLM field from Xiaomi.
Xiaomi has taken numerous steps to enhance internal organizational capabilities and attract external talent. In mid-November 2024, Xiaomi's Basic Technology Platform Department established an AI Platform Department, led by Duo Zhang, who has been publicly praised by Lei as "Xiaomi's great god."
Subsequently, Fuli Luo, one of the key developers of DeepSeek-V2, is expected to join Xiaomi, potentially working in the AI Lab. Renowned in the natural language processing (NLP) field, Luo's involvement in DeepSeek-V2, known for its significantly lower operational costs, has garnered attention. Her addition will further accelerate Xiaomi's R&D efforts in LLMs.
Xiaomi's LLM parameter scale is in the tens of billions. In contrast, Vivo launched its Blue Heart model with a parameter scale reaching hundreds of billions in early November. Consequently, there is limited public awareness of Xiaomi's LLM capabilities.
Nevertheless, Xiaomi has made notable progress in its self-developed LLM. In May, Xiaomi's MiLM successfully passed the large model filing process. By November, the MiLM2 series was released, continuing the "lightweight" philosophy with a parameter scale still in the tens of billions.
In fact, how to make AI LLMs deliver more practical functionalities rather than merely showcasing technology has become a challenge for various manufacturers. Industry analysis indicates that integrating AI features into smartphones has become an inevitable trend. The evolution of smartphones and operating systems towards enhanced compatibility and application of AI technologies undoubtedly represents a mainstream trend for the future.