Assignment 1 - Due September 28, 2018
- Consider two binary random variables X and Y with joint probability distribution p(x,y) given by
p(0,0) = 1/2, p(0,1) = 1/4, p(1,0) = 0, and p(1,1) = 1/4.
Find the values of H(X), H(Y), H(X|Y), H(Y|X), H(XY), and I(X;Y).
- A discrete memoryless source has symbols
x1, ..., x6 with p(x1) = 1/2 and p(x2) = 1/4.
Determine the upper and lower bounds on the entropy of this source.
A jar contains 5 black balls and 10 white balls.
Experiment X involves randomly drawing a ball out of the jar.
Experiment Y involves randomly drawing a ball with the ball drawn in experiment X not replaced in the jar.
One is interested in the colour of the drawn ball.
(a) How much uncertainty does experiment X contain?
(b) What is the uncertainty in experiment Y given that the first ball is black?
(c) What is the uncertainty in experiment Y given that the first ball is white?
(d) How much uncertainty does experiment Y contain?
- Let X be a random variable whose entropy H(X) is 8 bits. Suppose that
Y(X) is a deterministic function that takes on a different value for each
value of X.
(a) What is H(Y)?
(b) What is H(Y|X)?
(c) What is H(X|Y)?
(d) What is I(X;Y)?
(e) Suppose now that the deterministic function Y(X) is not invertible,
so that different values of X may correspond to the same value of Y(X).
In this case, what can be said about H(Y), and also about H(X|Y)?
- A Division Series is a five game baseball series held in October that ends
as soon as a team wins 3 games.
Define a random variable X that represents the outcome of
the series between two teams A and B.
Three possible values of X are BBB, ABAA, and BBAAA (many more outcomes can occur).
Let Y be the number of games played which range from 3 to 5.
Assuming teams A and B are equally matched and that the games are independent,
calculate H(X), H(Y), H(Y|X) and H(X|Y).
- In a country, 25% of the people are blond and 75% of all blond people have blue eyes.
In addition, 50% of the people have blue eyes.
How much information is received in each of the following cases:
(a) if we know that a person is blond and we are told the colour (blue/not blue)
of their eyes,
(b) if we know that a person has blue eyes and we are told the colour (blond/not
blond) of their hair,
(c) if we are told both the colour of their hair and that of their eyes.
Assignment 2 - Due 2018
Assignment 3 - Due 2018
Assignment 4 - Due 2018
Assignment 5 - Due 2018