Machine learning in signal processing WS 20/21This course is an introduction into statistical machine learning and artificial intelligence. The special emphasis is on applications to modern signal processing problems. The course is focused on design principles of machine learning algorithms. First we will study basic methods for regression and classification: linear regression, logistic regression, the nearest neighbors algorithm. Based on these examples, we will discuss the fundamental tradeoff between the flexibility of the model and the ability to fit the model based on the moderate amount of training data. We will contrast learning in highdimensional spaces vs. learning in low dimensional spaces. Next, we will study methods that help make linear models flexible: polynomial features and splines. When these tools are used, regularization is crucial. We will discuss structured signal representations: shorttime Fourier transform and wavelets. We will focus on the importance of sparsity in signal representations. This will lead us to compressed sensing and to other modern convexoptimizationbased methods for signal denoising, reconstruction, and compression. We will review key concepts in convex optimization, study the LASSO, support vector machines, the idea of kernels. The last part of the course will focus on the breakthrough new technology for computer vision: the deep learning. The course contains exercises: 40 percent mathematical and 60 percent implementing basic algorithms in Python. You will be asked to implement basic machine learning and signal processing algorithms yourself. For more advanced algorithms, you will practice using powerful numerical and optimization libraries (numpy, cvxpy, scikitlearn, pywavelets, pytorch). Database info is here. Learning goals are here. LiteratureWe follow:
Basic online course on probability and statistics:
Basic online course and books on linear algebra:
Suggested next steps:
TimeLectures:
Review sessions:
Office hours:
CommunicationTo be kept up to date, please register for the course on StudOn. Lecture handouts and videosIntroduction
Feature modeling
Convex optimizationbased methods
Neural networks
Discussion handouts
Problem sets
