Below is the tentative list of the topics that are available. Please, consider the specified literature as starting point for your literature research.
Topic Area 1: Machine Learning
- Computer Vision: Convolutional Neural Networks and Vision Transformers (Daniel Diefenthaler)
- Federated Learning (Fabian Dreer)
- Konečný et al., Federated Optimization: Distributed Machine Learning for On-Device Intelligence, https://arxiv.org/abs/1610.02527, 2016
- Yang et al., Federated Machine Learning: Concept and Applications, https://arxiv.org/abs/1902.04885, 2019
- Kairouz et al., Advances and Open Problems in Federated Learning, https://arxiv.org/abs/1912.04977, 2021,
- LEAF: A Benchmark for Federated Settings, https://leaf.cmu.edu/, 2019
- ML in Computational Sciences and HPC (Sergej Breitner)
- Fox et al., Understanding ML driven HPC: Applications and Infrastructure, https://arxiv.org/abs/1909.02363, 2021
- Fawzi et al., Discovering faster matrix multiplication algorithms with reinforcement learning, https://www.nature.com/articles/s41586-022-05172-4, 2022
- Li et al., Fourier Neural Operator for Parametric Partial Differential Equations, https://arxiv.org/abs/2010.08895, 2020
- Pestourie et al., Active learning of deep surrogates for PDEs: application to metasurface design, https://www.nature.com/articles/s41524-020-00431-2, 2021
- Karniadakis et al., Physics-informed machine learning, Nature Reviews, https://www.nature.com/articles/s42254-021-00314-5, 2021
- AI Sustainability (Karl Führlinger)
- Wu et al., Sustainable AI: Environmental Implications, Challenges and Opportunities, https://proceedings.mlsys.org/paper/2022/file/ed3d2c21991e3bef5e069713af9fa6ca-Paper.pdf, 2022
- Dodge et al., Measuring the Carbon Intensity of AI in Cloud Instances, https://arxiv.org/abs/2206.05229, 2022
- Strubell et al., Energy and Policy Considerations for Deep Learning in NLP, https://arxiv.org/pdf/1906.02243.pdf, 2019.
Topic Area 2: Generative AI
- Generative AI for Text: Transformers (Fabio Genz)
- Open AI, GPT-4 Technical Report, https://arxiv.org/abs/2303.08774
- Touvron et al., Llama 2: Open Foundation and Fine-Tuned Chat Models, https://ai.meta.com/research/publications/llama-2-open-foundation-and-fine-tuned-chat-models/
- Brown et al., Language Models are Few-Shot Learners, 2020
- Devlin et al., BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding, 2019
- Vaswani et al., Attention is all you need, https://arxiv.org/abs/1706.03762
- Generative Models for Image Generation (Michelle To)
- Rombach et al, High-Resolution Image Synthesis with Latent Diffusion Models, https://arxiv.org/abs/2112.10752, 2022.
- DALL-E: Creating Images from Text, https://openai.com/dall-e-2/, 2021
- Ramesh et al, Hierarchical Text-Conditional Image Generation with CLIP Latents, https://arxiv.org/abs/2204.06125, 2022
- Goodfellow et al., Generative Adversarial Nets, https://arxiv.org/abs/1406.2661, In Advances in Neural Information Processing Systems (NeurIPS), 2014
- Multimodel Generative AI (Sophia Grundner-Culemann)
- Benchmarking Generative AI (Minh Chung)
- Srivastava et al., Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models, https://arxiv.org/abs/2206.04615, 2023
- Liang et al., Holistic Evaluation of Language Models, https://arxiv.org/pdf/2211.09110.pdf, 2022
- Ali Borji, Pros and Cons of GAN Evaluation Measures: New Developments, https://arxiv.org/abs/2103.09396, 2021
- ML Commons, https://mlcommons.org/, 2023
Topic Area 3: AI Infrastructure
- Data management systems (Sophia Grundner-Culemann)
- Behm et al., Photon: A Fast Query Engine for Lakehouse Systems, SIGMOD, https://cs.stanford.edu/~matei/papers/2022/sigmod_photon.pdf, 2022
- Armbrust et al., Lakehouse: A New Generation of Open Platforms that Unify Data Warehousing and Advanced Analytics, CIDR, https://www.cidrdb.org/cidr2021/papers/cidr2021_paper17.pdf, 2021
- Armbrust et al., Delta Lake: High-Performance ACID Table Storage over Cloud Object Stores, https://www.databricks.com/wp-content/uploads/2020/08/p975-armbrust.pdf, 2020
- Vector Databases (Andre Luckow)
- Johnson et al., Billion-scale similarity search with GPUs, https://arxiv.org/abs/1702.08734, 2017
- Facebook AI Similarity Search (Faiss), https://faiss.ai, 2023
- Guo et al., Manu: A Cloud Native Vector Database Management System, https://arxiv.org/pdf/2206.13843.pdf, 2022
- Yi et al., Milvus: A Purpose-Built Vector Data Management System, https://arxiv.org/pdf/2107.10021.pdf, 2021
- Retrieval Augmented Generation (Andre Luckow)
- Scaling Machine Learning (Josef Pichlmeier)
- Zhao et al., PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallel, https://arxiv.org/pdf/2304.11277.pdf, 2023
- Narayanan et al., Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LM, https://arxiv.org/abs/2104.04473, 2021
- Hoffmann et al., Training Compute-Optimal Large Language Models (Chincilla), Deepmind, https://arxiv.org/pdf/2203.15556.pdf, 2022.
- Kaplan et al., Scaling Laws for Neural Language Models, https://arxiv.org/pdf/2001.08361.pdf, 2021
- Dean et al., Large Scale Distributed Deep Networks, 2012
- Alex Krizhevsky, One weird trick for parallelizing convolutional neural networks, 2014
- Li et al., Scaling Distributed Machine Learning with the Parameter Server, OSDI, 2014
- AI Hardware (Karl Führlinger)
- Hooker, The Hardware Lottery, 2020
- NVIDIA H100 Tensor Core GPU Architecture Overview, https://resources.nvidia.com/en-us-tensor-core/gtc22-whitepaper-hopper, 2022
- GraphCore, https://www.graphcore.ai/
- Jouppi et al., TPU v4: An Optically Reconfigurable Supercomputer for Machine Learning with Hardware Support for Embeddings, https://arxiv.org/abs/2304.01433, 2023
- Google TPU v5, https://cloud.google.com/blog/products/compute/announcing-cloud-tpu-v5e-and-a3-gpus-in-ga
- Cerabras, https://www.cerebras.net/
Topic Area 4: Quantum Computing
- Quantum Machine Learning (Florian Kiwit)
- Abbas, The power of quantum neural networks, https://arxiv.org/pdf/2011.00027.pdf, 2020
- Cerezo, Variational quantum algorithms, https://www.nature.com/articles/s42254-021-00348-9, 2021
- Lloyd et al., Quantum machine learning, https://www.nature.com/articles/nature23474, 2016
- Schuld et al., An introduction to quantum machine learning, https://arxiv.org/pdf/1409.3097.pdf
- Hubregtsen et al., Evaluation of Parameterized Quantum Circuits: on the relation
between classification accuracy, expressibility and entangling capability, 2020
- PennyLane, https://pennylane.ai/
- Quantum Benchmarking (Florian Krötz)
- Cross et al., Validating quantum computers using randomized model circuits, https://arxiv.org/abs/1811.12926, 2018
- Blume-Kohout et al., A volumetric framework for quantum computer benchmarks, https://arxiv.org/abs/1904.05546, 2020
- Mills et al., Application-Motivated, Holistic Benchmarking of a Full Quantum Computing Stack, https://arxiv.org/abs/2006.01273, 2021
- Martiel et al., Benchmarking quantum co-processors in an application-centric, hardware-agnostic and scalable way, https://arxiv.org/abs/2102.12973, 2021
- Lubinski et al., Application-Oriented Performance Benchmarks for Quantum Computing, https://arxiv.org/abs/2110.03137, 2021
- Finzgar et al., QUARK: A Framework for Quantum Computing Application Benchmarking, https://arxiv.org/abs/2202.03028, 2022.
- Quantum Chemistry (Korbinian Staudacher)