MAI 569: Information Theory in AI Systems

This course delves into the pivotal role of Information Theory in Artificial Intelligence (AI) systems, offering a deep dive into how principles of data transmission, compression, and entropy underpin the efficiency and effectiveness of AI technologies. Through engaging lectures and hands-on labs, students will explore the mathematical frameworks and algorithms that facilitate machine learning models' ability to learn from data, make predictions, and improve over time. The curriculum covers essential topics such as Shannon's entropy, information gain, and mutual information, applied in the context of optimizing neural networks, enhancing data encoding schemes, and ensuring secure AI communication channels. Designed for both theorists and practitioners, this course empowers participants with the knowledge to harness information theory concepts in developing advanced AI systems, optimizing their performance, and pioneering innovative solutions in the field.