CONCORDIA UNIVERSITY
Department of Electrical and Computer Engineering
ELEC 6151 - Information Theory and Source Coding

Summer 2013

Instructor:

Dr. M. R. Soleymani, Office: EV-5.125, Telephone: 848-2424 ext: 4103.

Time and Place: Monday, Wednesday, 17:45 – 20:15, Room H-523.

Office Hours: Monday, Wednesday, 14:00 – 15:00

Areas to be Covered:

• Entropy and Mutual Information.
• The Asymptotic Equipartition Property.
• Data Compression for Discrete Sources.
• Discrete Memoryless Channels and Capacity.
• Channel Coding Theorems.
• Waveform Channels.
• Rate Distortion Theory.
• Network Information Theory.

Text:

T.M. Cover and J.A. Thomas, Elements of Information Theory, Second Edition, Wiley, 2006.

Chapters 2, 3, 4, 5, 7, 8, 9, 10, 15 will be covered in detail.

References:

• Abbas El Gamal and Young-Han Kim, Network Information Theory, Cambridge Univ. Press, 2011.
• Imre Csiszar an Janos Korner, Information theory: coding theorems for discrete memoryless systems, Cambridge University Press, 2011 (Chapter 2).
• R.G. Gallager, Information Theory and Reliable Communications, John Wiley & Sons, 1968.
• R.E. Blahut, Principles of Information Theory, Addison Wesley, 1987.

Grading Scheme:

Assignments: 10%

Project: 20%

Two Class Tests: 20% (each 10%)

Final Examination: 50%

IMPORTANT NOTICE:

• In order to pass the course, you should get at least 60% in the final.
• The Quiz Tests and the Final exam will be open book. Only a text book (any text book covering the material of the course) will be allowed in the exam. No other material is allowed during the exams. No non-authorized copy of the text will be allowed in the class, in the class tests or in the final.
• Failing to write a class test results in losing the 10% assigned to the test.

COURSE NOTES

MIDTERM SAMPLES

FINAL SAMPLES

LIST OF ASSIGNMENTS:

Assignment 1: 2.3, 2.4, 2.5, 2.6, 2.10, 2.12, 2.13, 2.14, 2.28, 2.29, 2.33. Solution

Assignment 2: 3.1, 3.2, 3.6, 3.7, 4.3, 4.6, 4.7, 4.15. Solution

Assignment 3: 5.3, 5.5, 5.6, 5.13, 5.16, 5.20, 5.23, 5.26, 5.27. Solution_Part1, Solution_Part2

Assignment 4: 7.1, 7.2, 7.3, 7.5, 7.8, 7.10, 8.1, 8.4, 8.5, 8.8, 8.9. Solution Part 1, Solution Part 2

Assignment 5: 9.1, 9.2, 9.3, 9.5, 9.6, 9.7, 9.8, 9.15. Solution.

Assignment 6: 10.1, 10.3, 10.5, 10.6, 10.8, 10.9, 10.17. Solution.

Assignment 7: 15.1, 15.2, 15.4, 15.15, 15.17, 15.21, 15.22, 15.31. Solution.

Suggested Project Topics:

· Capacity Analysis of Correlated MIMO Channels.

· The Diversity-Multiplexing Tradeoff of the MIMO Relay Channel.

· Degrees of Freedom of the Network MIMO Channel.

· The Generalized Degrees of Freedom Region of the MIMO Interference Channel.

· Gaussian MIMO Broadcast Channel With Common and Private Messages.

· Interference Localization for MIMO Interference Networks.