Information Theory MCQ Quiz - Objective Question with Answer for Information Theory - Download Free PDF

Concept

The Channel Capacity Theorem, also known as the Shannon-Hartley theorem, is a fundamental principle in telecommunications and information theory. It determines the maximum rate at which information can be transmitted over a communication channel without error, given the channel's bandwidth and the signal-to-noise ratio.

According to the Shannon-Hartley theorem, the channel capacity is directly proportional to the channel bandwidth. This means that increasing the channel bandwidth will increase the channel capacity, allowing more information to be transmitted over the channel without error.

The theorem is expressed as:

\(C = B \log_2(1 + \frac{S}{N})\)

where:

• C is the channel capacity in bits per second (bps).
• B is the channel bandwidth in hertz (Hz).
• S is the average signal power.
• N is the average noise power.

Concept

Entropy is a measure of the uncertainty or randomness in a set of data or information. In the context of information theory, it quantifies the average amount of information produced by a stochastic source of data. For a source with M equiprobable symbols, the entropy (H) is calculated using the formula:

\(H = \log_2(M)\)

where M is the number of equiprobable symbols.

This is because each symbol has an equal probability of occurring, and the entropy measures the average amount of information per symbol. When all symbols are equiprobable, the entropy is simply the logarithm of the number of symbols to the base 2, which represents the average number of bits needed to encode each symbol.

Therefore, the correct answer is option 2.

 Key Points

• Grapevine is a casual business communication channel.
• Its name stems from the fact that it extends in all directions throughout the company, regardless of authority levels.
• The phrase "heard through the grapevine" refers to information obtained through rumors or gossip that is informal and unofficial.
• The typical interpretation is that the information was spread orally among friends or coworkers, sometimes in a private manner.
• Even if there are official channels in an organization, informal channels usually arise from interactions between members of the organization.

 Important Points

• To achieve goals, managers in informational jobs create, acquire, and disseminate knowledge with staff members and superior colleagues.
• All designs, including e-learning design, should adhere to the universal design concept of the hierarchy of information.
• Combining data from disparate sources with various conceptual, contextual, and typographic representations is known as information integration (II). 

Noise figure (NF) and noise factor (F) are measures of degradation of the signal-to-noise ratio (SNR), caused by components in a signal chain.

\(Noise\;Figure = \frac{{{{\left( {SNR} \right)}_{i/p}}}}{{{{\left( {SNR} \right)}_{o/p}}}}\)

In dB it is given as:

(N.F)dB = [(SNR)i/p]dB – [(SNR)o/p]dB

Important Points

In terms of noise resistance NF is given as:

\(NF= 1 + \frac{{{R_{eq}}}}{{{R_s}}}\) ------(1)

Req = Equivalent input resistance of the antenna

Rs = Noise resistance of the system

Faxes are specifically designed to transmit scanned images of documents over telephone lines.

Key Points

• The scanned image is converted into a signal that can be transmitted through the phone line and then converted back into an image at the receiving end.
• This makes them a good option for sending documents that need to be preserved in their original format, such as signed papers or contracts.

Hint

• Modem: A modem is a device that allows a computer to connect to the internet or another computer using a telephone line.
• Message: This is a broad term that could refer to any type of communication, including text, voice, or video. 
• E-mail: E-mail is a digital communication system that uses computers and networks to send and receive messages. 

Hence, the correct answer is Fax.

The correct option is 2

Concept:

Decibels are used to express the ratio of two power values. The decibel scale is a logarithmic scale, which is a more useful way to compare ratios than using a plain arit
hmetic scale.

The decibel (dB) is defined as 10 times the base 10 logarithm of the power ratio:

dB = 10 log(P2/P1)

To get the power ratio when the dB value is given, the formula is rearranged as:

P2/P1 = 10^dB/10

So for a change of 1 dB, the power ratio would be:

10 ^(1/10) = 1.25892 (approx)

This is about 26% increase in power level, since 1.25892-1 = 0.25892, which is roughly 26%.

So, 1 dB corresponds to approximately a 26% increase or decrease when you are increasing or decreasing power levels respectively.

Concept:

Mutual information of two random variables is a measure to tell how much one random variable tells about the other.

It is mathematically defined as:

I(X₁, X₂) = H(X₁) – H(X₁/X₂)

Application:

Since X₁ and X₂ are independent, we can write:

H(X₁/X₂) = H(X₁)

I(X₁,X₂ ) = H(X₁) – H(X₁)

= 0

The correct option is 3

Concept:

The maximum data rate of a noiseless channel is determined by the Nyquist formula, which states that the maximum data speed (in bits per second) is 2 x Bandwidth x log₂(L), where L is the number of signal levels.

For a binary channel, you only have two signal levels (0 and 1), so the log₂(L) part of the formula will be log₂(2) = 1

So, if you have a 2-kHz binary channel, you substitute the bandwidth value into the formula giving: 2 x 2000 Hz x 1 = 4000 bps.

Concept:

Information associated with the event is “inversely” proportional to the probability of occurrence.

Entropy: The average amount of information is called the “Entropy”.

\(H = \;\mathop \sum \limits_i {P_i}{\log _2}\left( {\frac{1}{{{P_i}}}} \right)\;bits/symbol\)

Rate of information = r.H

Calculation:

Given: r = 16 outcomes/sec

\(P_1=\frac{1}{2}\), and \(P_2=\frac{1}{64}\) 

\( H = \frac{1}{2}{\log _22} + \frac{1}{64}{\log_264};\)

\(H=\frac{19}{32} \ bits/outcomes\)

∴ Rate of information = r.H

R_s = 16 x 19/32

R_s = 19/2 or 38/4 bits/sec

Channel Capacity theorem: 

It states the channel capacity C, meaning the theoretical highest upper bound on the information rate of data that can be communicated at an arbitrarily low error rate using an average received signal power S through an analog communication channel that is subject to additive white Gaussian noise (AWGN) of power N.

The capacity of a band-limited AWGN channel is given by the formula:

\(C = B{\log _2}\left( {1 + \frac{S}{N}} \right)\)

C = Maximum achievable data rate with units of bits/sec for continuous nature of input and output. For discrete nature of input/output, bits/channel is used.

B = channel bandwidth

\(\frac{S}{N}\) = signal Noise power (in W)

Note: In the expression of channel capacity, S/N is expressed in Watts and not in dB.

The generator Matrix is given by

\(G = \left[ {{I_K}{P^T}} \right]\)

\({P^T} = \left[ {\begin{array}{*{20}{c}} 1&1&0\\ 0&1&1\\ 1&1&1\\ 1&0&1 \end{array}} \right]\)

The parity check matrix is given by:

H = [P I_{kn – K}]

Syndrome

S = eH^T

\({H^T} = \left[ {\begin{array}{*{20}{c}} {{P^T}}\\ {{I_{n - k}}} \end{array}} \right]\)

\({H^T} = \left[ {\begin{array}{*{20}{c}} 1&1&0\\ 0&1&1\\ 1&1&1\\ 1&0&1\\ 1&0&0\\ 0&1&0\\ 0&0&1 \end{array}} \right]\)

S = eH^T

For error in 7^th Bit

E = [000 0001]

\(S = \left[ {000\;000\;1} \right]\left[ {\begin{array}{*{20}{c}} 1&1&0\\ 0&1&1\\ 1&1&1\\ 1&0&1\\ 1&0&0\\ 0&1&0\\ 0&0&1 \end{array}} \right]\)

S = [ 0 0 1]

Extra information:

Syndrome for all possible errors

Entropy: It is the average amount of information that must be delivered in order to resolve the uncertainty about the outcome of a trial.

Or, The entropy of a probability distribution is the average or the amount of information when drawing from a probability distribution.

It is calculated as:

\(H = \mathop \sum \limits_{i = 1}^M {p_i}{\log _2}\frac{1}{{{p_i}}}\) bits

pi is the probability of the occurrence of a symbol.

The number of outcomes = M.

Notes:

Bandwidth: The bandwidth of a communication channel is defined as the difference between the highest and lowest frequencies that the channel allows passing through it (i.e., its passband).

Quantum: A quantum is defined as the minimum amount of any physical entity involved in an interaction. For example, a photon is a single quantum of light.

Concept:

Information associated with the event is “inversely” proportional to the probability of occurrence.

Mathematically, this is defined as:

\(I = {\log _2}\left( {\frac{1}{P}} \right)\;bits\)

P and I represent the probability and information associated with the event

Calculation:

With P = 1/4, the information associated with it will be:

\(I = {\log _2}\left( {\frac{1}{1/4}} \right)\;bits\)

\(I = {\log _2}\left( {4} \right)\;bits\)

I = log₂ (2²) bits

Since log_x yⁿ = n log_x y, the above can be written as:

I = 2 log₂(2)

I = 2 bits

Entropy: The average amount of information is called the “Entropy". It is given by:

\(H = \;\mathop \sum \limits_i {P_i}{\log _2}\left( {\frac{1}{{{P_i}}}} \right)\;bits/symbol\)

Concept:

Information associated with the event is “inversely” proportional to the probability of occurrence.

Mathematically, this is defined as:

\(I = {\log _2}\left( {\frac{1}{P}} \right)\;bits\)

P and I represent the probability and information associated with the event

Example: 

Sun rises in the east: The probability of this event to occur = 1, and the information associated with it will be = 0.

Sun rises in the west: P = 0 and I = ∞

Entropy: The average amount of information is called the “Entropy". It is given by:

\(H = \;\mathop \sum \limits_i {P_i}{\log _2}\left( {\frac{1}{{{P_i}}}} \right)\;bits/symbol\)

Noise figure (NF) and noise factor (F) are measures of degradation of the signal-to-noise ratio (SNR), caused by components in a signal chain.

\(Noise\;Figure = \frac{{{{\left( {SNR} \right)}_{i/p}}}}{{{{\left( {SNR} \right)}_{o/p}}}}\)

In dB it is given as:

(N.F)dB = [(SNR)i/p]dB – [(SNR)o/p]dB

Important Points

In terms of noise resistance NF is given as:

\(NF= 1 + \frac{{{R_{eq}}}}{{{R_s}}}\) ------(1)

Req = Equivalent input resistance of the antenna

Rs = Noise resistance of the system