Recent questions and answers in Information Theory

Consider a discrete memoryless source with an alphabet of four source symbols. $s(t)$ is a multi-level $(-1,0,+1,+2)$ signal representing a long sequence of random symbol...

A source transmits symbols from an alphabet of size $16$. The value of maximum achievable entropy (in bits) is $\_\_\_\_\_\_$.

The frequency of occurrence of $8$ symbols $(a-h)$ is shown in the table below. A symbol is chosen and it is determined by asking a series of $\text{"yes/no"}$ questions ...

Recall that the entropy (in bits) of a random variable $\mathrm{X}$ which takes values in $\mathbb{N}$, the set of natural numbers, is defined as$$H(X)=\sum_{n=1}^{\infty...

The capacity of a certain additive white Gaussian noise channel of bandwidth $1 \mathrm{~MHz}$ is $\mathrm{known}$ to be $8 \text{ Mbps}$ when the average transmit power ...

Consider a $2^{k} \times N$ binary matrix $A=\left\{a_{\ell, k}\right\}, a_{\ell, k} \in\{0,1\}$. For rows $i$ and $j$, let the Hamming distance be $d_{i, j}=\sum_{\ell=1...

Consider a channel where $x_{n} \in\{0,1\}$ is the input and $y_{n}=x_{n} * z_{n}$ is the output, where $*$ is $\text{EX-OR}$ operation, and $P\left(z_{n}=x_{n-1}\right)=...

Consider a discrete time channel with binary inputs and binary outputs. Let $x_{n}$ denote the input bit at time $n$ and $y_{k}$ denote the output bit at time $\text{k}$....

A source generates three symbols with probabilities $0.25,0.25,0.50$ at a rate of $3000$ symbols per second. Assuming independent generation of symbols, the most efficien...

A memoryless source emits $n$ symbols each with a probability $p$. The entropy of the source as a function of $n$increases as $\log n$decreases as $\log (1 / n)$increases...

Consider a Binary Symmetric Channel (BSC) with probability of error being $p$. To transmit a bit, say $1,$ we transmit a sequence of three $1\text{s}.$ The receiver will ...

Source encoding in a data communication system is done is order toenhance the information transmission ratereduce the transmission errorsconserve the transmitted powerfac...

A binary source has symbol probabilities $0.8$ and $0.2$. If extension coding (blocks of $4$ symbols) is used. The lower and upper bounds on the average code word length ...

An analog signal is band-limited to $4 \; \mathrm{kHz}$, sampled at the Nyquist rate and the samples are quantized into $4$ levels. The quantized levels are assumed to be...

The transition diagram of a discrete memoryless channel with three input symbols and three output symbols is shown in the figure. The transition probabilities are as mark...

Consider a real valued source whose samples are independent and identically distributed random variables with the probability density function, $f(x),$ as shown in the fi...

A speech signal, band limited to $4\:\text{kHz}$, is sampled at $1.25$ times the Nyquist rate. The speech samples, assumed to be statistically independent and uniformly d...

In a high school having equal number of boy students and girl students, $75\%$ of the students study Science and the remaining $25\%$ students study Commerce. Commerce st...

A binary communication system makes use of the symbols “zero” and “one”. There are channel errors. Consider the following events:$\begin{array}{ll} x_{0}& : & \text{a “z...

The capacity of a Binary Symmetric Channel $\text{(BSC)}$ with cross-over probability $0.5$ is ________.