GENERATING SERIES FOR ERROR STATISTICS OF BLOCK CODES ON CHANNELS WITH MEMORY

We present an analytical method for evaluating the performance of non-interleaved forward error correcting codes on channels that exhibit statistical dependence in the occurrence of errors. We consider a model for such a chan­ nel based on the probabilistic function of a Markov chain, also known as finite state channel (FSC) model. The main idea is to apply combinatorial methods to derive simple ex­ pressions for the probability of the number of symbol errors produced by the channel in terms of a coefficient in a formal power series. These methods are used to derive expressions for the codeword error probability of various practical cod­ ing schemes, including nonbinary block codes and the con­ catenation of two block codes. The general expressions are specialized for a Gilbert-Elliott channel with known model parameters, and numerical results are derived. Resumo Apresentamos um rnetodo analftico para calcu­ lar 0 desempenho de c6digos corretores de erro em canais que exibem dependencia estatistica na sequencia de erro. Con­ sideramos um modelo para tais canais baseado em funcoes probabilfsticas de uma Cadeia de Markov, tambem conhecido como modelo de canais de estados finitos (FSC). A ideia cen­ tral consiste em aplicar metodos enumerativos para derivar expressces para a probabilidade de erro em tennos de coe­ ficientes especfficos de urna serie de potencia formal. Este metodo e aplicado para calcular 0 desempenho de varies esquemas de codificacao, incluindo c6digos nao binaries e c6digos concatenados.


INTRODUCTION
The design of reliable communication systems has been an active area of research in the past few years.Among the dif ficulties encountered by the communication system designer to accomplish this goal, we stress the special system require ments (in general, high carrier frequencies, high data rates, low power consumption are highly desired and sometimes necessary), and the harsh channel environment.For exam ple, in a typical mobile communication scenario, propagation phenomena lead to the presence of signal arriving along difo.
ferent paths, each having random strength, phase and delay.With these system requirements and channel conditions, the combined effect of the channel impairments tend to introduce distortion in the transmitted process in such a way that the er rors in the received sequence are grouped together in clusters, called bursts.Because of the statistical dependence in the oc currence of errors, the channel is said to exhibit memory.
To achieve high reliability in data transmission over chan nels with memory, a good understanding of the channel char acteristics is of fundamental importance.The mathematical model for a discrete channel has three constituents: The set of possible discrete inputs to the channel, the set of possible quantized outputs, and the set of probabilities on a sequence of output symbols, conditional on the inputs.The error se quence produced by the channel is defined by some measure of difference between the input and output sequence.This error sequence incorporates all the physical impairments and distortions present in the channel.
The discrete channel is the so-called coded channel.The discrete inputs and outputs to the coded channel are, respec tively, the output sequence of the encoder {xd~l' and the input sequence to the decoder {ydk'=l' Thus, for example, in a binary system the error sequence will be a sequence of zeros and ones defined as follows: At the k-th time interval, the er ror bit ek is equal to zero (indicating no error) if Xk = Yko or ek is equal to one (indicating an error) if Xk =1= Yk.The char acteristics of the channel impairments, such as a correlated fading process, are incorporated into a mathematical model in such a manner as to produce a statistically similar error sequence as the output of the hard decision receiver.
One important family of discrete mathematical models that has been extensively used to characterize the error sequence is the family of finite state channel (FSC) models [1].De pending on the application, such models are also referred to as a hidden Markov model, function of Markov chains, and probabilistic automaton.An FSC model is characterized by an underlying non-observable Markov chain.The output symbol of the FSC at the k -th time interval, Yko is a func tion of the input symbol x k and the state of the Markov chain Sk.The channel is described statistically by the conditional probability P(Yk, Sk I Xk, Sk-l).Each state, Sko may be as sociated with a particular channel quality (for example, each state might represent a particular distribution of fading) and the transition of states simulates the time-varying character istic of the channel.
Several FSC models have been successively used to model practical communication systems, as we can cite: Tele phone networks [2], high-frequency microwave links [3,4], satellite channels [5], magnetic and optical recording sys tems [6], spread-spectrum frequency-hopped multiple access channels [7], the effect of fading in a mobile radio chan nel [8,9,10], communication systems with loss of synchro nization [11], and the burst nature of the Viterbi algorithm decoding [12].This paper is devoted to the application of the combina torial methods in the evaluation of the performance of for ward error correcting codes over channels modeled as gen eral FSC models.We follow the theory of enumeration of constrained sequences to enumerate a particular subset of er ror sequences.This powerful and general approach allows us to find a simple formula for the sum of a formal power series, the generating series, where the probabilities of interest are the coefficients of appropriate powers of the indeterminates.
These combinatorial methods are used to derive expres sions for the codeword error probability and bounds to sym bol and bit error probabilities of various practical coding schemes, including Reed-Solomon (RS) codes and the con catenation of two block codes.Methods for determining the codeword error probability for specific models, as for exam ple, Fritchman [6], and Gilbert-Elliott [2] are known.The contribution of this paper lies in presenting analytical results for the performance of non-interleaved error correcting codes over general FSC models, irrespective the number of states and the structure of the Markov chain.The results of this pa per are specialized for a Gilbert-Elliott channel with known model parameters, and numerical results are derived.The substance of this paper comes from work done in a doctoral's thesis [13].
This paper is organized into four sections.A brief review of FSC models and its properties is given in the next sec tion.The performance analysis of coding schemes over FSC models is the main concern of Section 3. Subsection 3.1 is dedicated to analyzing the performance of RS codes.This class of codes are of considerable importance in part because they have an efficient decoding algorithm based on their well understood algebraic structure.They also have optimum min imum distance since they are maximum distance separable codes.In Subsection 3.2 we investigate the performance of concatenated codes.Finally, Section 4 presents our conclu sions to this paper.

PROPERTIES OF FINITE STATE CHAN NELS
In this section we describe the details of the generation of the error process with FSC models and its properties.The statistical characterization of the error sequence generated by an FSC model will also be discussed.
The input sequence to the channel, denoted as {Xdk'=l' is transmitted across a binary channel and the correspond ing output sequence is {Yk}k'=l' The impairments due to the channel will be considered as an additive error sequence Consider {Sd ~o an N -state, homogeneous, first-order Markov chain with a finite state space N 1V = {G, 1, ... ,N I}.This Markov chain is characterized by a set of tran sition probabilities, Pi,j = P(Sk = j I Sk-l = i), i, j E JV.N , where Pi,j is independent upon a particular time interval k.Let P be an ]V x I"; transition probability ma trix, whose (i,j) entry is Pi,j.We say that a state j can be reached from a state i, if there exists some ri ~ Gsuch that Throughout this paper we consider a particular class of FSC models where the error sequence is generated as follows.At the k-th time interval, the chain makes a transition from state Sk-l = Sk-l to Sk = Sk with probability PSk-1,Sk and generates an output symbol ek E N 2 (independent of Sk-l), with probability bsk,ek = P(Ek = ek I Sk = Sk)' The sequence {edk'=l is known as the observed sequence of the model, and notice that the underlying state sequence cannot be determined from the observed sequence.We assume that the following definitions and properties are valid: • The error process is independent of the input sequence (channels that possess this property are called symmetric channels).Then where each error symbol ek is equal to the difference between the output and input symbols, i.e., ek = Yk ffi Xb since in N 2 S and e are the same.
• The distribution of the initial state is the stationary dis tribution II.Hence, the error sequence is a stationary process and is completely characterized when the mul tidimensional probability P(e n) f::,.P(ele2 ... en ) is known for all values of n.
Revista da Sociedade Brasileira de Telecornunicacoes Volume 13, numero 2, dezembro 1998 • When conditioned on the state process, the error process is memoryless, that is: Having established the basic properties of the model, we turn to the problem of calculating the probability of an error sequence.By the law of total probability and the previous definitions, the probability of an error sequence of length n., L::.en = el e2 ... en, may be expressed as: P( en I So = so) = = (2::: S n EN N n P( En = en I Sn = Sn, So = so) SnEN N k=l Hence N-l n P( en) = 2: 71"so 2: II bSk,ek PSk_1,Sk' (1)   so=O SnEN N k=l Equation (1) can also be written using matrix form [11]. De fine two N x N matrices, P (0) and P (1), where the (i, j) entry of the matrix P(ed is P(Ek = eu, Sk = j I Sk-l = i) = bj,ek Pi,j, which is the probability that the output sym bol is ek when the chain makes a transition from state i to i Equation (1) has a matrix form given by: The following expressions are valid for the matrices P and IT: P = P(O) + P(I); (3) (4) (5) >From ( 4) and ( 5) we conclude that the vectors ITT and 1 are the left and right eigenvectors of the matrix P, respec tively, corresponding to the eigenvalue 1.A binary stationary FSC model is completely specified by the matrices P (0) and P(I), since the initial distribution is the unique solution of Equation ( 4), and P is given by Equation (3).Various burst channel models have been proposed in the past few decades.The first model, based on a binary prob abilistic function of a Markov chain, was proposed in 1960 by Gilbert [15] in his studies to characterize the telephone channel.The Gilbert channel model, or, for short, the Gilbert channel, is a two-state Markov chain composed of an error free (good) and an error (bad) state.When the chain is in the good state the error bit ek is always zero.Otherwise, when it is in a bad state, the error bit is zero (no error) with prob ability 1 -b, or one (error) with probability b.The Gilbert channel was generalized by Elliott [2] in 1963, yielding the well known Gilbert-Elliott channel, by introducing a param eter g, which is the probability that the error bit is one (er ror) when the chain is in a good state.A further generaliza tion was proposed by McCullough [16], whose model dif fers from Gilbert-Elliott's channel essentially in two ways: (i) more than one bad state is considered, (ii) transition be tween states are permitted only immediately after an error.In spite of its simplicity, the derivation of expressions for cer tain error statistics can be surprising difficult and closed form expressions for quantities such as capacity, are not known at all.
A comprehensive presentation of many mathematical mod els proposed in the sixties and seventies is given in [1].In the following example we specialize the equations developed in this section for the Gilbert-Elliott channel.

EXAMPLE: THE GILBERT-ELLIOTT CHANNEL
The Gilbert-Elliott channel [2,15] consists of a two-state Markov chain.The state space is #2 = {O, I}. and bO,l = g, and b1,1 = b, as shown in Figure 1..The model has four pa rameters Q, q, b, g.By definition, 9 < b (the state 0 is the good state).The matrices P, P (0), P (1) and IT are given by: q (1g) (l-q)(I-b) , (1q) b ; (9) L 71"1 q+Q Some error distributions for this channel can be calculated using Equation (2).As an example, the probability the bit (II) • p • P(ll) n" P(l) P(l) In Equation (11) p is the good-to-bad ratio defined as: The performance evaluation of a coded communication system operating over an FSC model will be discussed in the next section.We will derive expressions for the probability of codeword error through the determination of P, (m, n), the probability of m channel error symbols in a block of length n symbols.A general model is described by a set of matrices Il , P (0), P (1) and the probability Ps (m, n) is expressed as a coefficient of particular terms in a formal power series.

PERFORMANCE ANALYSIS
We now establish some notation.On denotes the sequence of O's of length n, for example, O 2 = 00.Let R be the field of real numbers.If s and z are commutative indeterminates, [sk zn] P (s, z) denotes the coefficient of sk Z n in the formal power series P(s, z).Let R < Xo, Xl > be the set of poly nomials in non-commuting indeterminates Xo and Xl, that is, the set of all sums of finite non-commuting products of Xo and Xl with coefficients taken from R. It is easy to see that R < Xo, Xl > together with the operations of addition and multiplication is a ring.For convenience, this ring is denoted by R < Xo, Xl >.If [.n is a set of binary sequences oflength n, denote the generating series for [.n by: Fin = L x e1 x e2 ... x en ER<xo,Xl>, enE en (13) where x e i E {xo, Xl} and the indeterminates Xo and Xl mark an error bit produced by the channel.For example, if [.4 is the subset of Nt defined as [. 4 ={ one error in a block of length 4}, then [.4 = {1000,0l00,0010,0001}.According to the definition of Fen in (13) the generating series for the set [.4 is: Consider that the error sequence is generated by an N state binary FSC model defined in terms of the matrices n, P(O), P(l).>From Equations (2) the probability of the set [.4 defined above may be expressed as: P([.4) = lIT {P(1)P(0)3 +P(0)P( 1)P(0)2

+ P(O?P(l)P(O) +P(0)3P(1)}1.
Notice that P([.n) may be obtained from the generating se ries Fe; simply by replacing X e i by P (e.) and wrapping the vector n" around the front and 1 around the back.We can formalize this concept by defining the mapping: acting as a homomorphism to the whole of the ring R < Xo, Xl >.MN(R) is the ring of all N x N matrices with entries taken from R. The probability of the set En may be expressed very compactly as: (15) An expression for P(E n ) can be written down directly from the generating series Fen in non-commuting indeterminates by acting on Fen with the homomorphism 6., and then wrap ping n" and 1 around it.Therefore, the main step to find

P([.n) is to determine the generating series FEn'
We now develop an expression for P, (m, n) for two cases: RS codes and concatenated codes.

GF(2
An (n, k) nonbinary code defined over the Galois field C ) , where c is a positive integer, has codewords of!e:::g~l, n symbols, k information symbols, ti -k parity check sym bols, and code rate R; = kin.For example, an primitive RS code has length n = 2 C -1 symbols and can correct any combination of up to t = l(n -k) /2J error symbols within a codeword, where t denotes the error correcting capability of the code, and lX J is the greatest integer less than or equal to x.
We consider a coded communication system where nonbi nary transmitted symbols, taking values on GF( 2 C ) , are trans mitted across a binary channel.Each symbol in a transmitted codeword is corrupted by an additive error symbol ek, com posed of a sequence of c error bits statistically distributed ac cording to the FSC model.Each error symbol ek can also be regarded as an element from GF (2 C   ) , where each sequence of c bits is the vector-space representation of the corresponding field element.The k-th received symbol within a codeword is the sum ) , where the addition is over GF (2 C  ) .The transmitted symbol is received correctly (i.e., Zk = Ck) if the symbol ek is the sequence of C con secutive zeros, denoted by Dc.Otherwise, if ek =J Oc the transmitted symbol is received incorrectly, Due to this sym bol orientation, RS codes are suited to an environment where both burst and random errors occur.An expression for the probability of m erroneous received symbols in a block of length ti, Ps(m, n), is developed next.
Let F c and Fe denote the generating series for sets of error symbols ek that produce a correct and an erroneous received symbol, respectively.Then ) ), and let s mark the number of error symbols from the set enumerated by Fe in an error word.Then, from the mapping zx defined in Equation ( 14), and from Equation (15), Ps(m,n) may be expressed as: = [s7n zn]II T .6.(1z {x6 + s ((xa + Xl)" -X6)})-11 (18) = [s7n zn]II T (I -z{ P(O)C + s( P"p(O)")})-ll.(19) From Equation (19) it is simple to derive recurrence formulas for Ps(m, n), which provides a rapid computational scheme for the problem.For a specific FSC model, Ps(m, n) can be expressed as [sm zn]p(s, z), where pes, z) is the ratio of two polynomials in s and z.The denominator polynomial is responsible for the recurrence relation, and the numera tor polynomial defines the initial conditions.For example, it is easy to show that P s ( m, n) for a two-state Gilbert-Elliott channel satisfies a six-term recurrence formula.For example, the recurrence formula for c = 1 (binary code) is given by: This is a typical behavior of memoryless channels.Therefore, over the span of 127 x 7 = 889 bits, models with fl < 0.8 make sufficient state transitions to assure this "random" be havior.However, when the memory increases, fewer transi tions occur between states and long bursts are more likely.
As a consequence, P s(m,127) spreads out and decreases slowly with m.To give an example, the curves show that P s(0,127) = 0.36 for fl = 0.99.This is the probability of being in the good state during all 889 bit intervals and making no error, which is equal to, 1!"o ((1 -Q) (1 -g))889 = 0.36.
The contribution of any other state sequence to Ps(m, 127) is negligible.In the sequel we discuss how the amount of memory affects the codeword error probability of RS codes.We consider the case of bounded-distance decoding: Given a received word, the decoder selects a codeword which lies within Hamming distance t of the received word.If no such codeword exists a decoding failure is declared.The decoder decodes the received word as the correct (transmitted) code word if no more than t error symbols occur in one codeword.
The probability of codeword error (PCE) is defined as the probability of occurrence of received words with more than t erroneous symbols.Thus (20)   m=O m=t+1 It was also observed that for a fixed k, PCE is minimum for fl = 0.6.In fact, the curves show two distinct modes of behavior of PCE, depending upon the burst length.In the region of short bursts, say fl < 0.6, as the memory increases the error bits become more concentrated within bursts and affect fewer symbols in a codeword.Therefore, short bursts help the performance of RS decoders.On the other hand, in the region of high memory, say fl > 0.8, where long bursts occur, reliable communication is possible only with longer low rate codes.
In this section we have used PCE as a measure of perfor mance.The evaluation of other important measures will be discussed in the next subsection.

AN UPPER BOUND TO THE SYMBOL AND BIT ERROR PROBABILITIES
Error control strategies are typically compared on the basis of various probabilistic measures of performance.In addition to PCE, the symbol error probability P s , and the bit error prob ability Pb, are also important measures in evaluating system  performance [17].An exact expression for either P, or P b is in general difficult to evaluate, since it depends to a large extent on the operation of the decoding algorithm.Expres sions for Pi, are available in the literature for block codes on memoryless channels with known codeword weight distribu tion [18].In this section, a simple upper bound on Hand P, that does not depend upon the decoding operation is pre sented.
Let C = (Cl,""Cn), Z = (Zl, ... ,Zn), and c = (Cl' ... ,c n ) be the transmitted codeword, the received word, and the codeword chosen by the decoder, respectively.The average decoded-symbol error probability is defined by [19]: N and weight less than or equal to Ti. Formulas (26) and (17) have been used to calculate PCE of concatenated codes.Figure 3.3.depicts the performance of concatenated codes with the same overall code rate.The code rates selected are

CONCLUSIONS
We have addressed the problem of evaluating the codeword error probability of error control schemes on general FSC models.The main idea to find an expression for this mea sure for a specific coding scheme is to express the probabil ity of the correctable error pattern as a coefficient in a formal power series.All numerical plots presented in this paper were generated by first deriving recurrence formulas from matrix expressions.Bounds on symbol and bit error probabilities can also be derived using the expressions given in Subsec tion 3.2..
In the first part of Section 3 we have derived an expres sion for the codeword error probability of non-interleaved RS codes.The results presented in this section have been applied to investigate the code error-correcting capability necessary to achieve a desired performance.These results were ex tended to the case of concatenated codes with binary block codes as the inner code.The comparisons reported in this paper among different coding schemes are only valid for the specific channel parameters considered.Clearly, the analy sis can be repeated for any FSC model of interest, and other coding schemes can be treated as well with the machinery we have presented in this work.
{Ed ~l and the channel output at the k-th interval is repre sented as Y k = XkffiE k.For the cases of interest the alphabet of the symbols X b Yk and Ek is binary, N2 = {G, I}, and the symbol ffi denotes addition modulo 2.An error is said to occur at the k-th time interval whenever E k = 1.

= lL 9 +
error is a I, P(I) ~ P(E k = 1), and the probability of two consecutive ones, P(ll) ~ P(E k = 1, Ek+l = 1), are given Generating Series for Error Statistics of Block Codes on Channels with Memory by: P(l) = n" P(l) 1 = 710 9 + 711 b; IIp b.

2 n
where the indeterminates Xo and Xl mark an error bit (pro duced by the channel) equal to 0 or 1, respectively.The set of all error symbol patterns of any length is (1 -F c -Fe)-1 .Notice that P s (m, n) is equal to the probability that m error symbols from the set enumerated by Fe occur in a block of GF(consecutive error symbols.Let the indeterminate z mark the length of an error word (an n-tuple of error symbols over C

Figure 2 .
Figure 2. shows Ps(m, n), as a function of m, for Gilbert Elliott channels with various values of memory, fl = (1 q -Q), for n = 127, and c = 7.Throughout this section we consider the following model parameters, p = q/Q = 20, b = 0.4, 9 = 0.001.The parameters Q and q are uniquely determined from fl and p.Because the Gilbert-Elliott channel has a parameter that can be interpreted as the memory of the channel, the effectiveness of coding schemes under several memory conditions can be evaluated.The average number of erroneous symbols in a received word of length ti is Tis = n ITT ( P" -p (O)C) 1 .Exam ples of values of Tis are 12 for fl = 0.6, 10 for fl = 0.8, 8 for fl = 0.92,7 for fl = 0.96.The curves of Figure 2. show that for values of fl approximately less than 0.8 the probabil ity Ps(m, 127) has a maximum roughly centered around Tis.

Figure 3 . 2 .Figure 2 .
Figure 3.2.shows PCE for RS codes with fixed length n, versus the memory fl, for various values of k (number of information symbols).In this analysis we consider PCE = 10-6 the required error probability for reliable communica tion.We can conclude from these plots that for a particular value of u, say fl = 0.92, PCE equals to 10-6 is achieved with rate R; = 49/127 = 0.39 for c = 7 (n = 127), and R; = 141/255 = 0.55 for c = 8 (n = 255).1 10-1

Figure 3 .
Figure 3. PCE versus memory u, for RS codes over Gilbert-Elliott channels having the number of information symbols k as a parameter, for C = 7 (a); C = 8 (b).

n 1 L 22 ) n 1 n
( ) E{N e } P, = -P Cj =I-Cj = ---, (21) n n j=l where the random variable N; is the number of symbols where c and c differ, and E{•} stands for the expected value of a random variable.If the random variable N; is the num ber of symbol errors in the received word, then (;: L E{Ne n, = m}Ps(m,n).(23) I m=t+l For the case of a t-error correcting bounded-distance decod ing, a simple upper bound to E{ N; I N; = m} is given by the following worst-case argument: Whenever t or more er rors occur, the received word lies within a sphere of a wrong (not transmitted) codeword, and that the decoder introduces t additional errors.Clearly, the maximum value assumed by Generating Series for Error Statistics of Block Codes on Channels with Memory [14]n is classified as irre ducible if, and only if, every state can be reached from every other state[14].We will now define the period of a state.A Si, = i) = G except when ti is multiple of d(i), and d(i) is the largest inte ger with this property.A chain is aperiodic when all its states have period equal to 1.We consider only irreducible and ape riodic Markov chains.For irreducible and aperiodic Markov chains, there exists a limiting distribution for Sko as k -7 00, Istate i has period d(i) when P(Sk+n = i independent of the starting distribution for So.This limit ing distribution is known as the stationary distribution and is denoted by the matrix II = [TiO Til ... 71" N_l]T, where the superscript [.jT indicates the transpose of a matrix.Addition ally, if the initial state So has distribution II, the subsequent states Sk have distribution II for all k.