Bisimulation for labelled markov processes citeseerx. Metrics for markov processes prakash panangaden1 1school of computer science mcgill university fall 2016, simons institute. As such the domain of interest seems far from traditional concurrency theory. It is very pleasing that one can obtain the axiomatisation of. Analysis of probabilistic systems boot camp lecture 5. Approximation techniques for lmps can be based on metrics 15,19,42 and coalgebras 43,44. Labelled markov processes are probabilistic labelled transition systems. Labelled markov processes are continuousstate fully probabilistic labelled transition systems.
This talk resumes a joint work of mine with dexter kozen cornell univ. Markov processes university of bonn, summer term 2008 author. Labelled markov processes mcgill cs mcgill university. In the context of finite markov decision processes mdps, we have built on these metrics to provide a robust quantitative analogue of stochastic bisimulation n. Probabilistic model checking of labelled markov processes via. This metric has the property that processes are at zero distance if and only if they are bisimilar. Jos ee desharnais, vineet gupta, radha jagadeesan and prakash panangaden.
Probabilistic model checking of labelled markov processes. Intuitively the smaller the distance between two processes, the more alike their behaviour. Labelled markov processes are probabilistic versions of labelled. Panangaden labelled markov processes friday, june 11, 2010 4. They can be seen as coalgebras of a suitable monad on the category of measurable space. The theory as developed so far included a treatment of bisimulation, logical characterization of bisimulation, weak bisimulation, metrics, universal domains for. Thus, markov processes are the natural stochastic analogs of the deterministic processes described by differential and difference equations. Almost sure bisimulation in labelled markov processes. We prove that the relation of bisimilarity between countable labelled transition systems lts is. The metric analogue of weak bisimulation for probabilistic processes.
Labelled markov processes as generalised stochastic relations. Panangadenbisimulation for labelled markov processes. The book covers basic probability and measure theory on continuous state spaces and then develops the theory of lmps. However, until now it has been unclear how to implement it in practice. Labelled markov processes are probabilistic versions of labelled transition systems with. Our results subsume existing results on completeness of probabilistic modal logics for markov processes. From now on, we shall only consider markov processes from this functional point of view. Markov decision processes and labelled concurrent markov chains with. Metrics for labelled markov processes prakash panangaden1 1school of computer science mcgill university january 2008, winter school on logic, iit kanpur panangaden labelled markov processes. Panangaden mcgill analysis of probabilistic systems metrics 9 33. This paper concerns labelled markov processes lmps, probabilistic models over uncountable state spaces originally introduced by prakash panangaden and colleagues.
An intrinsic characterization of approximate probabilistic. There are various stochastic extensions of modelling formalisms for hybrid systems, e. Panangaden consider the probabilistic choice operator. This space does not appear explicitly in the study of labelled markov processes but one does see. Markov processes or labelled markov processes 18 are just a very special instance of these where there is a set of unary operators for the transitions.
Introduction bisimulation implies logical agreement measure theory the gory details concluding remarks the easy direction let r be a bisimulation relation on an lmp s. Markov processes consider a dna sequence of 11 bases. In an lmp, instead of one transition probability function or markov kernel there are several, each associated with a distinct label1. Desharnais, gupta, jagadeesan and panangaden generalized. Prakash panangaden is an americancanadian computer scientist noted for his research in programming languages, concurrency theory, markov processes and duality. Introduction bisimulation implies logical agreement measure theory the gory details concluding remarks the easy direction let r. Labelled markov processes are probabilistic labelled transition systems where the state space is not necessarily discrete. By author prakash panangaden mcgill university, canada. A set of possible world states s a set of possible actions a a real valued reward function rs,a a description tof each actions effects in each state. Essays dedicated to prakash panangaden on the occasion of his 60th birthday. When the process starts at t 0, it is equally likely that the process takes either value, that is p1y,0 1 2. An approximation algorithm for labelled markov processes. A markov process is a random process in which the future is independent of the past, given the present. Introduction discrete probabilistic transition systems labelled markov processes probabilistic bisimulation simulation.
Labelled markov processes are probabilistic versions of labelled transition systems. In this paper we introduce a new class of labelled transition systemslabelled markov processes and define bisimulation for them. Labelled transition systems prakash panangaden1 1school of computer science mcgill university january 2008, winter school on logic, iit kanpur panangaden labelled markov processes. Labelled markov processes world scientific publishing co. Approximating labelled markov processes sciencedirect.
They form one of the most important classes of random processes. Precup, proceedings of uai04, auai press, arlington, va, 2004, pp. Probabilistic model checking of labelled markov processes 43 propositions that hold in that state, and a reward structure r. Pdf bisimulation for labelled markov processes prakash. Bisimulation metrics for continuous markov decision processes. In proceedings of 17th annual ieee symposium on logic in computer science, pages. Labelled markov processes probabilistic bisimulation simulation the need for measure theory basic fact. In traditional exact reasoning, the best that one can do is to say that all these three. This paper concerns labelled markov processes lmps, prob abilistic models over uncountable state spaces originally introduced by. Introduction labelled markov processes a tutorial overview prakash panangaden school of computer science mcgill university 12th june 2014, cornell university panangaden mcgill labelled markov processes mfps xxx 1 41. Approximating labelled markov processes request pdf. Expressiveness of probabilistic modal logics, revisited.
In general, the state space of a labelled markov process may be a continuum. Labelled markov processes lmps were formulated bdep97, dep02 to study such general interacting markov processes. Metrics for labelled markov processes sciencedirect. Labelled markov processes labelled markov processes lmps panangaden et al. In general, the state space of a labelled markov process may be a. Jagadeesan and panangaden 6 has very appealing theoretical properties. Bisimulation metrics for continuous markov decision.
Prakash panangaden labelled markov processes world of. Bisimulation for labelled markov processes r blute, j desharnais, a edalat, p panangaden proceedings of twelfth annual ieee symposium on logic in computer science, 1997. If this is plausible, a markov chain is an acceptable. Then, sa, c, g, t, x i is the base of positionis the base of position i, and and x i i1, 11 is ais a markov chain if the base of position i only depends on the base of positionthe base of position i1, and not on those before, and not on those before i1. Approximate notions of probabilistic bisimilarity are formally characterised and computed for nitestate labelled markov processes in 17. Bisimulation for labelled markov processes sciencedirect.