ECE 515
Information Theory
Assignments
Assignment 1  Due September 28, 2018
 Consider two binary random variables X and Y with joint probability distribution p(x,y) given by
p(0,0) = 1/2, p(0,1) = 1/4, p(1,0) = 0, and p(1,1) = 1/4.
Find the values of H(X), H(Y), H(XY), H(YX), H(XY), and I(X;Y).
 A discrete memoryless source has symbols
x_{1}, ..., x_{6} with p(x_{1}) = 1/2 and p(x_{2}) = 1/4.
Determine the upper and lower bounds on the entropy of this source.

A jar contains 5 black balls and 10 white balls.
Experiment X involves randomly drawing a ball out of the jar.
Experiment Y involves randomly drawing a ball with the ball drawn in experiment X not replaced in the jar.
One is interested in the colour of the drawn ball.
(a) How much uncertainty does experiment X contain?
(b) What is the uncertainty in experiment Y given that the first ball is black?
(c) What is the uncertainty in experiment Y given that the first ball is white?
(d) How much uncertainty does experiment Y contain?
 Let X be a random variable whose entropy H(X) is 8 bits. Suppose that
Y(X) is a deterministic function that takes on a different value for each
value of X.
(a) What is H(Y)?
(b) What is H(YX)?
(c) What is H(XY)?
(d) What is I(X;Y)?
(e) Suppose now that the deterministic function Y(X) is not invertible,
so that different values of X may correspond to the same value of Y(X).
In this case, what can be said about H(Y), and also about H(XY)?
 A Division Series is a five game baseball series held in October that ends
as soon as a team wins 3 games.
Define a random variable X that represents the outcome of
the series between two teams A and B.
Three possible values of X are BBB, ABAA, and BBAAA (many more outcomes can occur).
Let Y be the number of games played which range from 3 to 5.
Assuming teams A and B are equally matched and that the games are independent,
calculate H(X), H(Y), H(YX) and H(XY).
 In a country, 25% of the people are blond and 75% of all blond people have blue eyes.
In addition, 50% of the people have blue eyes.
How much information is received in each of the following cases:
(a) if we know that a person is blond and we are told the colour (blue/not blue)
of their eyes,
(b) if we know that a person has blue eyes and we are told the colour (blond/not
blond) of their hair,
(c) if we are told both the colour of their hair and that of their eyes.
Assignment 2  Due 2018
Assignment 3  Due 2018
Assignment 4  Due 2018
Assignment 5  Due 2018
Aaron Gulliver
20180914