(a) A psychological subject can make one of two responses A₁ and A2. Associated with these responses are a set of N stimuli {S₁, S₂, ..., S_N}. Each stimulus is conditioned to one of the responses. A single stimulus is sampled at random (all possibilities equally likely) and the subject responds according to the stimulus sampled. Reinforcement occurs at each trial with probability Ï€ (0 < Ï€="">< 1)="" independent="" of="" the="" previous="" history="" of="" the="" process.="" when="" reinforcement="" occurs,="" the="" stimulus="" sampled="" does="" not="" alter="" its="" conditioning="" state.="" in="" the="" contrary="" event="" the="" stimulus="" becomes="" conditioned="" to="" the="" other="" response.="" consider="" the="" markov="" chain="" whose="" state="" variable="" is="" the="" number="" of="" stimuli="" conditioned="" to="" response="" ax.="" determine="" the="" transition="" probability="" matrix="" of="" this="" m.c.="">

 

(b) A subject S can make one of three responses A₀, A₁, and A₂. The A₀ response corresponds to a guessing state. If S makes response A₁, the experiment reinforces the subject with probability Ï€₁ and at the next trial S will make the same response. If no reinforcement occurs (probability 1 — Ï€₁), then at the next trial S passes to the guessing state. Similarly n₂ is the probability of reinforcement for response A₂. Again the subject remains in this state if reinforced and otherwise passes to the guessing state. When S is in the guessing state, he stays there for the next trial with probability 1 — c and with probabilities c/2 and c/2 makes responses A₁ and A₂ respectively. Consider the Markov chain of the state of the subject and determine its transition probability matrix.