Αποτελέσματα Αναζήτησης
In information theory, the conditional entropy quantifies the amount of information needed to describe the outcome of a random variable given that the value of another random variable is known. Here, information is measured in shannons, nats, or hartleys.
Definition The conditional entropy of X given Y is. H(X|Y ) = −. p(x, y) log p(x|y) = −E[ log(p(x|y)) ] (5) The conditional entropy is a measure of how much uncertainty remains about the random variable.
do we define the entropy of a random variable? What is entropy? I. Entropy is an important notion in thermodynamics, information theory, data compression, cryptography, etc.
The joint entropy \(H(X, Y)\) of a pair of discrete random variables \((X, Y)\) with a joint distribution \(p(x, y)\) is defined as \[H(X, Y) = - \sum_{x \in \mathcal{X}} \sum_{y \in \mathcal{Y}} p(x, y) \log p(x, y) = - E \log p(X, Y)\]
Conditional entropy. Let (X;Y) ∼ p. For x ∈ Supp(X), the random variable YSX = x is well defined. The entropy of Y conditioned on X, is defined by. H(YSX) ∶= E H(YSX = x) = E H(YSX) x←X X. Measures the uncertainty in Y given X. Let pX & pYSX be the marginal & conational distributions induced by p. H(YSX) = Q pX(x) ⋅ H(YSX = x) x∈X.
conditional information, and relative entropy (discrimination, Kullback-Leibler information), along with the limiting normalized versions of these quantities such as entropy rate and information rate.
The conditional entropy is a measure of the uncertainty of X, given that we know the value of Y. From: Quantum Mechanics with Applications to Nanotechnology and Information Science, 2013
