AI Integration: Large Models Enter Space for the First Time
During this year's National Day holiday, the news about the "world's first in-orbit operation of AI large model technology verification" has attracted people's attention.
Xinhua News Agency reported on October 6th that Chengdu Guoxing Aerospace Technology Co., Ltd. (hereinafter referred to as "Guoxing Aerospace") has recently conducted in-orbit operation tests of satellite AI large models, covering different temperature conditions and various types of reasoning and answering, and the mission has been successfully completed.
The reporter learned that using AI large models to empower the satellite industry has become a consensus in the industry. "Before the AI large model, the standard process of satellite remote sensing was to send the data obtained after the satellite was launched to the ground receiving station, and then the receiving station would preprocess the data, that is, process it into a continuous picture or a picture that meets the analysis requirements, and then manually find the desired information." Sha Aijun, a senior practitioner in the field of satellite remote sensing, told the reporter that when the number of satellites increases, it is impossible to operate business-like solely by humans, and it is unstable.
Advertisement
Sun Haijiang, director of the Image Department of the Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, once said that with the increase in satellite scale, the global daily acquisition of observation data has been measured in petabytes, and traditional manual and single-domain remote sensing information extraction methods are difficult to adapt to the rapid interpretation requirements of massive remote sensing data.
The reporter noticed that companies and research rooms, including Ant Group, Alibaba Damo Academy, SenseTime Group, and Pengcheng Laboratory supported by Huawei, are all developing remote sensing large models. In late September, the world's first hundred billion parameter-level remote sensing basic model "Aerospace·Lingmu" 3.0, jointly developed by the scientific research team of the Institute of Aerospace Information Innovation, Chinese Academy of Sciences, and Pengcheng Laboratory, was released. It is not only China's first generative basic model for multimodal remote sensing data but also the first industry basic model specifically created for the remote sensing field.
"From our own development strategy, (AI large model on satellite) is a very important link, and it is also the start of our subsequent development ladder." Guoxing Aerospace CEO Wang Lei explained the significance of the large model on the satellite to the reporter in this way. Sha Aijun believes that using large models on satellites may solve the following problems: "The first is the autonomous control problem of satellites, that is, the flight safety problem of satellites, which can be regarded as the execution problem that satellites need to solve for automatic driving, and the second is the problem of information extraction when collecting ground information."
It is understood that at 10:31 on September 24th this year, the world's first AI large model scientific satellite jointly developed by Guoxing Aerospace and the Chinese University of Hong Kong was successfully launched into orbit by the Jie Long III carrier rocket in Haiyang, Shandong.
At 20:46 on September 25th, the test team sent AI tasks to the satellite through the ground station in the form of remote control commands; at 21:11, the satellite successfully started running the AI large model in the sky above the North Atlantic with high-performance computing load on board, and the AI large model's space in-orbit operation process and results were displayed in real-time through the satellite's own in-orbit visualization evidence system - the "Star Screen" system; at 22:20, the test team received the complete data of the first test through the ground station.
From September 25th to October 5th, the satellite conducted a total of 13 AI large model in-orbit operation tests, covering different temperature conditions and various types of reasoning and answering, and successfully completed all the predetermined targets of the AI large model technology verification in the satellite's in-orbit operation.The success of this technical experiment also signifies the global first successful in-orbit operation of AI large model technology verification for satellites.
Journalists observed that during the in-orbit operation test period, the satellite AI large model responded to a series of questions such as "What superpower would you choose?", "What do you want to say to the motherland on National Day?", and "What are the recommended travel destinations for National Day?". The official website shows that Guoxing Aerospace is a leading domestic AI satellite internet technology company, founded by former university research institutes and internet industry leaders in the field of satellite internet. To date, Guoxing Aerospace has completed 13 space missions.
GuoXing Aerospace has formed B-side, G-side, and C-side satellite internet products for different application scenarios through a low-cost rapid response satellite development technology system, a full-stack AI satellite network technology system, and a high-maneuverability rapid revisit satellite technology system, serving hundreds of government and enterprise customers.
According to Qichacha, Guoxing Aerospace was established in May 2018 and has gone through five rounds of financing. The most recent round of financing occurred in 2023, with an amount exceeding 500 million yuan, led by Hongtai Fund, followed by investments from Taishan Urban Construction Group, Qingdao Haifa Group, Jiaxing Jiaxiu Group, Ceyuan Capital, and Wuhu Industrial Investment.
In Guoxing Aerospace's view, what are the main functions of large models on satellites?
In response, Guoxing Aerospace stated: "By leveraging the on-board computing payload, we carry out in-orbit reasoning and training technology verification of AI large models, to accumulate technical experience for 'Tian Shu Tian Suan' and the next step of 'Di Shu Tian Suan', and to promote the commercialization process of AI integration between the sky and the ground in China and globally."
Link bandwidth is the biggest bottleneck.
It is reported that "Tian Shu Tian Suan" refers to the data obtained by satellites in space without the need to be transmitted back to the ground, and the calculation can be completed in space, relying on the computing payload of AI satellites and the inexhaustible solar energy; "Di Shu Tian Suan" refers to transmitting the computing requirements from the ground to the computing nodes in space, and performing data calculations through the computing payload of AI satellites.
In addition, Guoxing Aerospace has a more ambitious "journey". Guoxing Aerospace stated, "Artificial intelligence and satellite internet are the super tracks with the most intense competition and rapid development between China and the United States on Earth and in space." China urgently needs to seize the current window of opportunity where there is a huge gap between China and the United States in terms of constellation networking scale and carrying capacity, but the single-satellite capabilities have not yet formed a generational difference. We need to develop AI large model satellites at the fastest speed, with the greatest effort, and with unconventional investment, "to take the lead in building a space-based computing power network, to win the initiative for China in the new round of information revolution and international competition, and to achieve a change of lanes to overtake."
In Sha Aijun's view, the so-called remote sensing large models or on-board computing, on-board intelligence, is directly saying that they want to move the ground-based cloud computing to the sky, but the reality is far from meeting the actual demand."The main issue with space-based computation is the high cost or insufficient efficiency of transmitting data from space to the ground," said Sha Ajun. "Whether it's cloud computing in space or onboard computation, the biggest bottleneck is essentially the bandwidth of the communication link from space to Earth. In other words, there are not enough opportunities for communication between the heavens and the Earth."
However, onboard AI still has room for commercial application imagination. "For example, after satellites collect remote sensing images, clients may not need the entire satellite image data, but only the number and location information of the ships within it. This information can be directly sent to mobile phones in the form of short messages," Sha Ajun pointed out. Applications such as shipping, insurance, and futures are concerned with the number and scale of ships in port, the volume of cargo, and the accumulation of goods at port terminals, which can be statistically used with some models and algorithms to predict price trends.
The key lies in whether there are barriers to advantage.
Although large models on satellites can be compared to large models on mobile phones, there are differences. "In fact, the difference between things on satellites and on the ground mainly lies in the following two issues: one is the issue of space radiation, which requires some electronic equipment to be reinforced before it can be used; the second is the power consumption issue. On the ground, you can stack hardware wildly, and miracles can be achieved with great effort, but the power supply of satellites mainly relies on solar panels, which means there is an upper limit," Sha Ajun said.
According to Sha Ajun, FPGA chips are currently the mainstream hardware for providing onboard AI computing power, and uploading the commonly mentioned CPUs and GPUs from the ground to satellites is a trend. He said that there are several foreign startups doing onboard computation, with a staff size of about five or six people. Their methods are basically to reinforce or encapsulate AI chips from companies like Intel, turning them into standard parts, and then selling them to satellite companies for use.
For example, if a material company can do a good job of electronic shielding, encapsulate Nvidia's GPU, and then send it to space with a satellite.
As for large models, Sha Ajun believes that the field of remote sensing actually has a large number of open-source models to learn from, including multimodal large models. "The problems that remote sensing needs to solve are a few: one is target recognition, the second is classification, and the third is change recognition," he said, pointing out that as long as there are a large number of sample data, target recognition can be done well.
As for why large models are used on satellites, Sha Ajun said that the field of satellite remote sensing definitely cannot escape the demand for large models, and the information extraction during satellite autonomous driving and ground information collection is the two scenarios he believes are most likely to be realized.
In December 2023, some research institutions and commercial aerospace companies successively launched artificial intelligence remote sensing large models, attracting attention in the industry. In 2023, the Aerospace Institute of the Chinese Academy of Sciences released the Satlas model, and Fudan University also released the GRAFT model. The scale of the model's data and parameters is getting larger and larger, and the performance is getting stronger and stronger.
At this year's World Artificial Intelligence Conference (2024 WAIC), Wang Jian, the person in charge of Ant Group's remote sensing large model, introduced that Ant Group has developed a 2 billion-parameter multimodal remote sensing model SkySense based on the Ant Bailing large model platform. Through technological innovations in data, model architecture, and unsupervised pre-training algorithms, SkySense can contribute technical capabilities to seven common remote sensing perception tasks such as land use monitoring and geomorphological change detection.Looking at the development trends of remote sensing models, there are three major trends: from supporting single-modality data to integrating multi-modality data; from only covering images from a single data source to being able to integrate images from multiple data sources; and from only supporting the interpretation of a single static image to integrating information from the entire time series of images.
Sha AiJun pointed out that although the industry is currently talking about satellites plus large models, there may not be many who are truly investing. In his view, the key to this investment lies in whether there are barriers to entry. "When doing it, you need to find out what can be done, and some barriers can be easily broken by large companies. I think the most important thing might be the barriers of data and samples, which are the core parts." He used his own professional experience to say that many satellite companies have a large amount of data, but they only store the data in databases without cleaning or annotating it. "Whether the data can become one-to-one corresponding samples, mastering data and mastering samples are not the same concept."