Information Theory
2023. Fall
 

Class meetings:

Monday 12:15 -15:00 (first time on September 4, 2022) location: lecture room IE218

Lecturer: Marta Pinter (email:marti(a)cs.bme.hu)

Topical information will be placed in the Moodle page

Textbooks and resources:


Lecture notes:

First lecture (Sep. 4, 2023)

Second lecture

Third lecture

Corresponding chapters from the book T. Cover, J. Thomas : Elements of Information Theory that may help: 2.1 ; 5.1 ; 5.2 ; 5.5 ; 2.6 (it uses the notion of relative entropy defined in 2.3. I don't use it in the notes.) ; 5.9 ; 5.11

Fourth lecture

Fifth lecture

Corresponding chapters from the book T. Cover, J. Thomas : Elements of Information Theory that may help: 2.2 ; 2.3 ; 2.5 ; 2.6 ; 5.3 ; 5.4 ; 5.6 ; 5.7 ; 5.8

Sixth lecture

Corresponding chapters from the book T. Cover, J. Thomas : Elements of Information Theory that may help: 4.1 ; 4.2 ; 4.4 ; 5.4

Seventh lecture

Corresponding chapters from the book T. Cover, J. Thomas : Elements of Information Theory that may help: 12.10

Eighth lecture

(Corresponding chapters from the book T. Cover, J. Thomas : Elements of Information Theory that may help: (13.1))

Ninth lecture : First part Entropy of the uniform quantizer

Tenth lecture : Problems, home works, Channel coding

Corresponding chapters from the book T. Cover, J. Thomas : Elements of Information Theory that may help: 8.1 ; 8.2 ; 8.3 ; 8.5

Eleventh lecture

Corresponding chapters from the book T. Cover, J. Thomas : Elements of Information Theory that may help: 8.7 ; 8.8 ; 8.9 ; 8.10

Consultations:

First consultation: September 25, 13:00 (1 p.m.) room: IE218

Second consultation: October 9, 13:00 (1 p.m.) room: IE218

Third consultation: October 30, 13:00 (1 p.m.) room: IE218

Fourth consultation: November 13, 13:00 (1 p.m.) room: IE218

Fifth consultation: November 27, 13:00 (1 p.m.) room: IE218


Midterms:

There will be two midterms during the semester scheduled below. The duration of both midterms will be 90 minutes. They will consist of two "theoretical" questions and four exercises. Given the nature of the first two questions the midterms are closed book and closed notes that is neither book nor notes are allowed to be used when writing them.

As a sample midterm, here is the first midterm of an earlier semester. The file contain sketches of solutions of the exercises but not on the first page which contains only the problems. It is suggested to look at solutions only after you have solved or at least tried to solve the exercises yourself. And a sample midterm for the second midterm. This file also contains sketches of solutions of the exercises, so again, try to solve the exercises yourself, and look at solutions only after you have tried.

1st midterm: October 16, 13:00 (1 p.m.), room: IE218

2nd midterm: December 4, 13:00 (1 p.m.) room: IE218


Grading:

For getting a valid grade at least 40% of the score should be achieved (separately) on both midterms. The final grade is calculated by averaging the results of the two midterms both taken into account with weight 1/2. Conversion to grades is as follows: from 80% to 100% performance the grade is 5. Below that it is 4 from 70%, 3 from 55%, and 2 from 40%. Below 40% no valid grade is obtained.


Make up tests:

There will be one separate make up test for both midterms.

Retake of 1st midterm: November 6, 13:00 (1 p.m.) room: IE218

Retake of 2nd midterm: December 12, 8:00 room: IE220

Rules of how make up midterm scores are taken into account:

If someone takes a make up midterm for improving the result of the corresponding midterm, then the new result will be valid even if it is weaker than the original one. There is one exception to this: If on an original midterm one already achieved 40% of the score then the midterm remains valid (counted with a minimum 40% score) even if the make up score is below 40%.