Funktionen

Below is the tentative list of the topics that are available. Please, consider the specified literature as starting point for your literature research.

Topic Area 1: Machine Learning

1. Computer Vision: Convolutional Neural Networks and Vision Transformers (Daniel Diefenthaler)
• Dosovitskiy, An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale, https://arxiv.org/abs/2010.11929, 2020
• Liu et al., A ConvNet for the 2020s https://arxiv.org/pdf/2201.03545.pdf, 2022
• Krizhevsky et al., ImageNet Classification with Deep Convolutional Neural Networks, 2012
2. Federated Learning (Fabian Dreer)
• Konečný et al., Federated Optimization: Distributed Machine Learning for On-Device Intelligence, https://arxiv.org/abs/1610.02527, 2016
• Yang et al., Federated Machine Learning: Concept and Applications, https://arxiv.org/abs/1902.04885, 2019
• Kairouz et al., Advances and Open Problems in Federated Learning, https://arxiv.org/abs/1912.04977, 2021,
• LEAF: A Benchmark for Federated Settings, https://leaf.cmu.edu/, 2019
3. ML in Computational Sciences and HPC (Sergej Breitner)
• Fox et al., Understanding ML driven HPC: Applications and Infrastructure, https://arxiv.org/abs/1909.02363, 2021
• Fawzi et al., Discovering faster matrix multiplication algorithms with reinforcement learning, https://www.nature.com/articles/s41586-022-05172-4, 2022
• Li et al., Fourier Neural Operator for Parametric Partial Differential Equations, https://arxiv.org/abs/2010.08895, 2020
• Pestourie et al., Active learning of deep surrogates for PDEs: application to metasurface design, https://www.nature.com/articles/s41524-020-00431-2, 2021
• Karniadakis et al., Physics-informed machine learning, Nature Reviews, https://www.nature.com/articles/s42254-021-00314-5, 2021
4. AI Sustainability (Karl Führlinger)
• Wu et al., Sustainable AI: Environmental Implications, Challenges and Opportunities, https://proceedings.mlsys.org/paper/2022/file/ed3d2c21991e3bef5e069713af9fa6ca-Paper.pdf, 2022
• Dodge et al., Measuring the Carbon Intensity of AI in Cloud Instances, https://arxiv.org/abs/2206.05229, 2022
• Strubell et al., Energy and Policy Considerations for Deep Learning in NLP, https://arxiv.org/pdf/1906.02243.pdf, 2019.

Topic Area 2: Generative AI

5. Generative AI for Text: Transformers (Fabio Genz)
• Open AI, GPT-4 Technical Report, https://arxiv.org/abs/2303.08774
• Touvron et al., Llama 2: Open Foundation and Fine-Tuned Chat Models, https://ai.meta.com/research/publications/llama-2-open-foundation-and-fine-tuned-chat-models/
• Brown et al., Language Models are Few-Shot Learners, 2020
• Devlin et al., BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding, 2019
• Vaswani et al., Attention is all you need, https://arxiv.org/abs/1706.03762
6. Generative Models for Image Generation (Michelle To)
   • Rombach et al, High-Resolution Image Synthesis with Latent Diffusion Models, https://arxiv.org/abs/2112.10752, 2022.
   • DALL-E: Creating Images from Text, https://openai.com/dall-e-2/, 2021
   • Ramesh et al, Hierarchical Text-Conditional Image Generation with CLIP Latents, https://arxiv.org/abs/2204.06125, 2022
   • Goodfellow et al., Generative Adversarial Nets, https://arxiv.org/abs/1406.2661, In Advances in Neural Information Processing Systems (NeurIPS), 2014
7. Multimodel Generative AI (Sophia Grundner-Culemann)
   • Radford et al., Learning Transferable Visual Models From Natural Language Supervision, https://arxiv.org/abs/2103.00020, 2021
   • Driess et al., PaLM-E: An embodied multimodal language model, Google Research, https://palm-e.github.io/assets/palm-e.pdf
   • Liu et al., Visual Instruction Tuning, https://arxiv.org/pdf/2304.08485.pdf
8. Benchmarking Generative AI (Minh Chung)
   • Srivastava et al., Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models, https://arxiv.org/abs/2206.04615, 2023
   • Liang et al., Holistic Evaluation of Language Models, https://arxiv.org/pdf/2211.09110.pdf, 2022
   • Ali Borji, Pros and Cons of GAN Evaluation Measures: New Developments, https://arxiv.org/abs/2103.09396, 2021
   • ML Commons, https://mlcommons.org/, 2023

Topic Area 3: AI Infrastructure

9. Data management systems (Sophia Grundner-Culemann)
   • Behm et al., Photon: A Fast Query Engine for Lakehouse Systems, SIGMOD, https://cs.stanford.edu/~matei/papers/2022/sigmod_photon.pdf, 2022
   • Armbrust et al., Lakehouse: A New Generation of Open Platforms that Unify Data Warehousing and Advanced Analytics, CIDR, https://www.cidrdb.org/cidr2021/papers/cidr2021_paper17.pdf, 2021
   • Armbrust et al., Delta Lake: High-Performance ACID Table Storage over Cloud Object Stores, https://www.databricks.com/wp-content/uploads/2020/08/p975-armbrust.pdf, 2020
10. Vector Databases (Andre Luckow)
• Johnson et al., Billion-scale similarity search with GPUs, https://arxiv.org/abs/1702.08734, 2017
• Facebook AI Similarity Search (Faiss), https://faiss.ai, 2023
• Guo et al., Manu: A Cloud Native Vector Database Management System, https://arxiv.org/pdf/2206.13843.pdf, 2022
• Yi et al., Milvus: A Purpose-Built Vector Data Management System, https://arxiv.org/pdf/2107.10021.pdf, 2021
11. Retrieval Augmented Generation (Andre Luckow)
• Lewis et al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks, https://arxiv.org/abs/2005.11401v4, 2020
• LangChain, https://python.langchain.com/docs/get_started/introduction.html, 2023
• Semantic Kernel, https://github.com/microsoft/semantic-kernel, 2023
12. Scaling Machine Learning (Josef Pichlmeier)
• Zhao et al., PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallel, https://arxiv.org/pdf/2304.11277.pdf, 2023
• Narayanan et al., Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LM, https://arxiv.org/abs/2104.04473, 2021
• Hoffmann et al., Training Compute-Optimal Large Language Models (Chincilla), Deepmind, https://arxiv.org/pdf/2203.15556.pdf, 2022.
• Kaplan et al., Scaling Laws for Neural Language Models, https://arxiv.org/pdf/2001.08361.pdf, 2021
• Dean et al., Large Scale Distributed Deep Networks, 2012
• Alex Krizhevsky, One weird trick for parallelizing convolutional neural networks, 2014
• Li et al., Scaling Distributed Machine Learning with the Parameter Server, OSDI, 2014
13. AI Hardware (Karl Führlinger)
    • Hooker, The Hardware Lottery, 2020
    • NVIDIA H100 Tensor Core GPU Architecture Overview, https://resources.nvidia.com/en-us-tensor-core/gtc22-whitepaper-hopper, 2022
    • GraphCore, https://www.graphcore.ai/
    • Jouppi et al., TPU v4: An Optically Reconfigurable Supercomputer for Machine Learning with Hardware Support for Embeddings, https://arxiv.org/abs/2304.01433, 2023
    • Google TPU v5, https://cloud.google.com/blog/products/compute/announcing-cloud-tpu-v5e-and-a3-gpus-in-ga
    • Cerabras, https://www.cerebras.net/

Topic Area 4: Quantum Computing

14. Quantum Machine Learning (Florian Kiwit)
    • Abbas, The power of quantum neural networks, https://arxiv.org/pdf/2011.00027.pdf, 2020
    • Cerezo, Variational quantum algorithms, https://www.nature.com/articles/s42254-021-00348-9, 2021
    • Lloyd et al., Quantum machine learning, https://www.nature.com/articles/nature23474, 2016
    • Schuld et al., An introduction to quantum machine learning, https://arxiv.org/pdf/1409.3097.pdf
    • Hubregtsen et al., Evaluation of Parameterized Quantum Circuits: on the relation between classification accuracy, expressibility and entangling capability, 2020
    • PennyLane, https://pennylane.ai/
15. Quantum Benchmarking (Florian Krötz)
    • Cross et al., Validating quantum computers using randomized model circuits, https://arxiv.org/abs/1811.12926, 2018
    • Blume-Kohout et al., A volumetric framework for quantum computer benchmarks, https://arxiv.org/abs/1904.05546, 2020
    • Mills et al., Application-Motivated, Holistic Benchmarking of a Full Quantum Computing Stack, https://arxiv.org/abs/2006.01273, 2021
    • Martiel et al., Benchmarking quantum co-processors in an application-centric, hardware-agnostic and scalable way, https://arxiv.org/abs/2102.12973, 2021
    • Lubinski et al., Application-Oriented Performance Benchmarks for Quantum Computing, https://arxiv.org/abs/2110.03137, 2021
    • Finzgar et al., QUARK: A Framework for Quantum Computing Application Benchmarking, https://arxiv.org/abs/2202.03028, 2022.
16. Quantum Chemistry (Korbinian Staudacher)
    • Cao et al., Quantum Chemistry in the Age of Quantum Computing, https://pubs.acs.org/doi/10.1021/acs.chemrev.8b00803, 2022
    • Quantum Algorithms for Quantum Chemistry and Quantum Materials Science, https://pubs.acs.org/doi/10.1021/acs.chemrev.9b00829, 2020
    • Huggins et al., Unbiasing fermionic quantum Monte Carlo with a quantum computer, https://arxiv.org/abs/2106.16235, 2022