Prof. Weijia Jia now is servingas the Director of AI and Future Networking Research Institute of Beijing Normal University (BNU, Zhuhai), Guangdong, China. He has served as the VP for Research atBNBU (UIC). Prior joing BNU, he served as the Deputy Director of StateKay Laboratory of Internet of Things for Smart City atthe University of Macau(2018-2020) and was Zhiyuan Chair Professor at Shanghai Jiaotong University (2014-2018). He worked in CityUniversity of Hong Kong as a professor (1995-2103). His contributions have been reconganized as intelligent edge computing and LLM applications, optimal network routing and deployment; vertex cover; anycast and QoS routing, and sensors networking; knowledge relation extractions and NLP. He has over 600 publications in the prestige international journals/conferencesand research books and book chapters. He has received many science-tech awards and served as area editor for various prestige international journals, chair and PC member/keynote speaker for top international conferences. He is the Fellowof IEEE and the Distinguished Member of CCF.

贾维嘉教授现任北京师范大学(珠海校区)人工智能与未来网络研究院院长。加入北京师范大学之前，他于2018–2020年在澳门大学担任智慧城市物联网国家重点实验室副主任，并于2014–2018年在上海交通大学担任至远讲席教授。1995–2013年，他曾在香港城市大学任教。贾教授在智能边缘计算与大模型应用、最优网络路由与部署、顶点覆盖、任播与QoS路由、及传感器网络、知识关系抽取与自然语言处理等领域做出了重要贡献。他在国际顶级期刊、会议以及学术专著和章节中发表论文600余篇。曾获得多项科技奖项，并担任多种国际著名期刊的分区编辑，以及国际顶级会议的主席、程序委员会委员和特邀报告人。他是IEEE会士及中国计算机学会（CCF）杰出会员。

Speech Title: Bridging Edges and LLMs

Abstract: Large Language Models (LLMs) are widely usedacross various domains, but deploying them in cloud datacenters often leads to significant response delays and high costs,undermining Quality of Service (QoS) at the network edge.Although caching LLM request results at the edge using vectordatabases can greatly reduce response times and costs for similarrequests, this approach has been overlooked in prior research. Toaddress this, we propose a novel Vector database-assisted cloud-Edge collaborative LLM QoS Optimization (VELO) frameworkthat caches LLM request results at the edge using vectordatabases, thereby reducing response times for subsequent similar requests. Unlike methods that modify LLMs directly, VELOleaves the LLM’s internal structure intact and is applicableto various LLMs. Building on VELO, we formulate the QoSoptimization problem as a Markov Decision Process (MDP)and design an algorithm based on Multi-Agent ReinforcementLearning (MARL). Our algorithm employs a diffusion-basedpolicy network to extract the LLM request features, determiningwhether to request the LLM in the cloud or retrieve results fromthe edge’s vector database. Implemented in a real edge system,our experimental results demonstrate that VELO significantlyenhances user satisfaction by simultaneously reducing delaysand resource consumption for edge users of LLMs. Our DLRSalgorithm improves performance by 15.0% on average for similarrequests and by 14.6% for new requests compared to thebaselines.