Tutorial Abstract

Machine learning techniques are now integral to the advancement of intelligent urban services, playing a crucial role in elevating the efficiency, sustainability, and livability of urban environments. The recent emergence of foundation models such as ChatGPT marks a revolutionary shift in the fields of machine learning and artificial intelligence. Their unparalleled capabilities in contextual understanding, problem solving, and adaptability across a wide range of tasks suggest that integrating these models into urban domains could have a transformative impact on the development of smart cities. Despite growing interest in Urban Foundation Models (UFMs), this burgeoning field faces challenges such as a lack of clear definitions, systematic reviews, and universalizable solutions.

In this tutorial, we will first introduce the concept of UFM and discuss the unique challenges involved in building them. We then present a data-centric taxonomy that categorizes and clarifies current UFM-related works, based on urban data modalities and types. Furthermore, to foster advancement in this field, we present a promising framework aimed at the prospective realization of UFMs, designed to overcome the identified challenges. Additionally, we explore the application landscape of UFMs, detailing their potential impact in various urban contexts. Relevant papers and open-source resources have been collated and are continuously updated at: https://github.com/usail-hkust/Awesome-Urban-Foundation-Models.

Tutorial Materials and Outline

Our [survey paper]

Tutorial Outline

1. Introduction [15 mins][slides]

We introduce the basic concepts and definitions related to Urban Foundation Models (UFMs) and how they can pave the way to Urban General Intelligence (UGI).

2. Challenges of Building UFMs [15 mins][slides]

This section discusses the challenges to building UFMs, including multi-source, multi-granularity, and multi-modal data integration; spatio-temporal reasoning capability; versatility to diverse urban task domains; and privacy and security concerns.

3. Overview of UFMs [110 mins][slides]

We introduce a data-centric taxonomy for UFMs-related studies to shed light on the progress and efforts made in this field. Based on the urban data modalities, we categorize the existing works on UFMs into six classes:

• Language-based models
• Vision-based models
• Trajectory-based models
• Time series-based models
• Multimodal models
• Others We will clarify these studies through the lens of their focused pre-training and adaptation techniques.

4. Prospects of UFMs and Future Directions [30 mins][slides]

To foster advancement in this field, we present a promising framework aimed at the prospective realization of versatile UFMs, designed to overcome the identified challenges.

Short Bio of Lecturers and Contributors

Hao Liu is currently an assistant professor at the Artificial Intelligence Thrust, Hong Kong University of Science and Technology (Guangzhou). Prior to that, he was a senior research scientist at Baidu Research and a postdoctoral fellow at HKUST. He received the Ph.D. degree from the Hong Kong University of Science and Technology (HKUST), in 2017. His general research interests are in data mining, machine learning, and big data management, with a special focus on mobile analytics and urban computing. He has published prolifically in refereed journals and conference proceedings, such as TKDE, VLDBJ, KDD, NeurIPS, VLDB, SIGIR, WWW, AAAI, and IJCAI.

Hui Xiong is a Chair Professor, Associate Vice President (Knowledge Transfer), and Head of the Artificial Intelligence Thrust at Hong Kong University of Science and Technology (Guangzhou). His research interests span artificial intelligence, data mining, and mobile computing. He obtained his PhD in Computer Science from the University of Minnesota, USA. Dr. Xiong has served on numerous organization and program committees for conferences, including as Program Co-Chair for the Industrial and Government Track for the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), Program Co-Chair for the IEEE 2013 International Conference on Data Mining (ICDM), General Co-Chair for the 2015 IEEE International Conference on Data Mining (ICDM), and Program Co-Chair of the Research Track for the 2018 ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. He received several awards, such as the 2021 AAAI Best Paper Award and the 2011 IEEE ICDM Best Research Paper award. For his significant contributions to data mining and mobile computing, he was elected as a Fellow of both AAAS and IEEE in 2020.

Weijia Zhang is currently a Ph.D. student at the Artificial Intelligence Thrust, Hong Kong University of Science and Technology (Guangzhou). His research interests include spatio-temporal data mining, urban computing, and time series. His works on urban intelligence have been published in several prestigious conferences and journals, such as KDD, ICML, WWW, VLDB, AAAI, ICDM, and TKDE.

Jindong Han is currently a Ph.D. student at the Hong Kong University of Science and Technology. His research interests include spatiotemporal data mining, urban computing, and large language models. He has published several research papers in prestigious conferences and journals, such as KDD, VLDB, AAAI, TKDE, and VLDBJ. He received the first prize award in the Fresh Air competition of KDD Cup 2018.

Zhao Xu received his B.E. degree from Tsinghua University. He is currently an MPhil student at the Hong Kong University of Science and Technology (Guangzhou). His works have been published by influential conferences and journals such as KDD, IEEE TMC and ACM SIGSPATIAL, aiming at developing reliable, trustworthy large models for real-world applications.

Hang Ni received his B.E. degree from Northwestern Polytechnical University. He is currently a Ph.D. candidate at Artificial Intelligence Thrust, Hong Kong University of Science and Technology (Guangzhou). His research interests lie in areas of graph learning and urban computing. Some of his works have been accepted by top conferences and journals such as KDD, CIKM and KBS.

Cite Our Work

If you find our work useful, please cite our work:

• Survey paper (Urban Foundation Models: A Survey)

  @inproceedings{ufmsurvey-kdd2024,
  author = {Zhang, Wejia and Han, Jindong and Xu, Zhao and Ni, Hang and Liu, Hao and Xiong, Hui},
  title = {Urban Foundation Models: A Survey},
  year = {2024},
  booktitle = {Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining}
  }