Recent questions tagged information-theory

An input to a $6$-level quantizer has the probability density function $f(x)$ as shown in the figure. Decision boundaries of the quantizer are chosen so as to maximize th...

A memoryless source emits $n$ symbols each with a probability $p$. The entropy of the source as a function of $n$increases as $\log n$decreases as $\log (1 / n)$increases...

Consider a Binary Symmetric Channel (BSC) with probability of error being $p$. To transmit a bit, say $1,$ we transmit a sequence of three $1\text{s}.$ The receiver will ...

A communication channel with AWGN operating at a signal to noise ratio $S N R \gg 1$ and bandwidth $B$ has capacity $C_{1}$. If the $S N R$ is doubled keeping $B$ constan...

Source encoding in a data communication system is done is order toenhance the information transmission ratereduce the transmission errorsconserve the transmitted powerfac...

A binary source has symbol probabilities $0.8$ and $0.2$. If extension coding (blocks of $4$ symbols) is used. The lower and upper bounds on the average code word length ...

An analog signal is band-limited to $4 \; \mathrm{kHz}$, sampled at the Nyquist rate and the samples are quantized into $4$ levels. The quantized levels are assumed to be...

Let $H(X)$ denote the entropy of a discrete random variable $X$ taking $K$ possible distinct real values. Which of the following statements is/are necessarily true?$H(X) ...

A symbol stream contains alternate $\text{QPSK}$ and $\text{16-QAM}$ symbols. If symbols from this stream are transmitted at the rate of $1$ mega-symbols per second, the ...

The transition diagram of a discrete memoryless channel with three input symbols and three output symbols is shown in the figure. The transition probabilities are as mark...

Consider communication over a memoryless binary symmetric channel using a $(7, 4)$ Hamming code. Each transmitted bit is received correctly with probability $(1 – \in),$ ...

Consider a channel over which either symbol $x_{A}$ or symbol $x_{B}$ is transmitted. Let the output of the channel $Y$ be the input to a maximum likelihood $\text{(ML)}$...

A speech signal, band limited to $4\:\text{kHz}$, is sampled at $1.25$ times the Nyquist rate. The speech samples, assumed to be statistically independent and uniformly d...

A digital transmission system uses a ($7,4$) systematic linear Hamming code for transmitting data over a noisy channel. If three of the message-codeword pairs in this cod...

In a high school having equal number of boy students and girl students, $75\%$ of the students study Science and the remaining $25\%$ students study Commerce. Commerce st...

A binary communication system makes use of the symbols “zero” and “one”. There are channel errors. Consider the following events:$\begin{array}{ll} x_{0}& : & \text{a “z...

Let $\left ( X_{1},X_{2} \right )$ be independent random variables. $\left ( X_{1} \right )$ has mean $0$ and variance $1$, while $\left ( X_{2} \right )$ has mean $1$ an...