Topic 3: Model Update and Dimension Enhancement Fusion Technology

Focusing on the key scientific issues of cloud co evolution of large and small models and efficient aggregation of multi end updates in systems, this study focuses on the separation and matching methods of multimodal slices of heterogeneous models. It aims to achieve breakthroughs in multimodal slice clipping, unreliable model transmission, and low-cost dimensionality fusion, forming a set of efficient update aggregation methods for end side heterogeneous models, and proposing adaptive perception technology for end side software and hardware environments; Propose a model multimodal slicing separation, cropping, and adaptation method for heterogeneous end side devices; Propose an adaptive model transmission protocol for dynamic network environments; Propose an efficient fusion method for massive heterogeneous models to achieve linear aggregation of models. Through the above research, we aim to address the core issue of limited adaptation of distributed terminal resources in the collaborative evolution of small and large model end clouds.

Main research content:

As shown in Figure 1. To address the issue of low efficiency in updating and integrating end-to-end models, research is conducted on ① multimodal slice clipping: a multimodal slice separation method is proposed, and a mechanism for eliminating end-to-end heterogeneous adaptive redundant slices is constructed to minimize model increment; ② Non reliable model transmission: Propose an algorithm to measure the importance of similar model parameters, design a rate adaptive fault-tolerant transmission protocol, and achieve low latency model incremental transmission; ③ Low cost dimensionality aggregation: Design a dimensionality increasing mechanism for isomorphic mapping of heterogeneous models to achieve linear convergence aggregation.