Eigil Fjeldgren Rischel 2024 5 25 https://erischel.com/efr-0002/ efr-0002 /efr-0002/ Bidirectionality in graphical models Eigil Fjeldgren Rischel 2024 5 25 https://erischel.com/efr-0003/ efr-0003 /efr-0003/ Structural Causal Model Definition Let be a (finite) directed acyclic graph (DAG). Then a structural causal model (SCM) on consists of For each vertex , measurable spaces A family of independent random variables ---these are called the exogenous variables For each , a measurable function (note that if has no parents, the product is just a singleton) There is a significant existing literature (see Abstracting causal models, Reference , Reference . For a survey see Reference ) studying transformations between structural causal models. These have generally been called something like abstractions, the typical examples being the relationship between a high-level and a low-level model of the same system. However, the precise properties that we should ask for in such a transformation have turned out to be somewhat subtle. The most obvious condition to impose is to ask, for any possible variable we could intervene on, say , and any other variable (or collection of variables) , that the diagram of kernels commutes---in other words, given an intervention in the low-level model, the two possible distributions in the high-level model agree. The main problem with this condition is that it is simply too strict. We generally can't ask that every low-level intervention is well-modeled by the corresponding high-level intervention. Rather, each high-level intervention corresponds to some distribution of the corresponding low-level variables, and we must ask that a diagram of this form: commutes. For example, if the low-level variables are the velocities of all the gas molecules in a container, and the high-level variables are thermodynamic quantities like temperature and pressure, we can't expect that every set of velocities corresponding to a given temperature will lead to a given distribution of outcomes---rather, intervening on the temperature leads to some particular distribution of possible sets of velocities, and this leads to the observed distribution of outcomes. The key here is that control information flows in the opposite direction from measurement information. This phenomenon repeats in many examples. References Fabio Massimo Zennaro 2022 7 18 https://erischel.com/zennaro_abstraction_2022/ zennaro_abstraction_2022 /zennaro_abstraction_2022/ Abstraction between structural causal models: A review of definitions and properties Reference 10.48550/arXiv.2207.08603 http://arxiv.org/abs/2207.08603 Eigil Fjeldgren Rischel Sebastian Weichwald 2021 8 5 https://erischel.com/rischel_compositional_2021/ rischel_compositional_2021 /rischel_compositional_2021/ Compositional abstraction error and a category of causal models Reference 10.48550/arXiv.2103.15758 http://arxiv.org/abs/2103.15758 Eigil Fjeldgren Rischel Sebastian Weichwald 2021 8 5 Abstract Interventional causal models describe several joint distributions over some variables used to describe a system, one for each intervention setting. They provide a formal recipe for how to move between the different joint distributions and make predictions about the variables upon intervening on the system. Yet, it is difficult to formalise how we may change the underlying variables used to describe the system, say moving from fine-grained to coarse-grained variables. Here, we argue that compositionality is a desideratum for such model transformations and the associated errors: When abstracting a reference model M iteratively, first obtaining M' and then further simplifying that to obtain M'', we expect the composite transformation from M to M'' to exist and its error to be bounded by the errors incurred by each individual transformation step. Category theory, the study of mathematical objects via compositional transformations between them, offers a natural language to develop our framework for model transformations and abstractions. We introduce a category of finite interventional causal models and, leveraging theory of enriched categories, prove the desired compositionality properties for our framework. Sander Beckers Joseph Y. Halpern 2019 7 9 https://erischel.com/beckers_abstracting_2019/ beckers_abstracting_2019 /beckers_abstracting_2019/ Abstracting causal models Reference http://arxiv.org/abs/1812.03789 Paul K. Rubenstein Sebastian Weichwald Stephan Bongers Joris M. Mooij Dominik Janzing Moritz Grosse-Wentrup Bernhard Schölkopf 2017 7 4 https://erischel.com/rubenstein_causal_2017/ rubenstein_causal_2017 /rubenstein_causal_2017/ Causal consistency of structural equation models Reference 10.48550/arXiv.1707.00819 http://arxiv.org/abs/1707.00819 Context Eigil Fjeldgren Rischel 2024 11 19 https://erischel.com/efr-003Z/ efr-003Z /efr-003Z/ Thesis Cutting Board Eigil Fjeldgren Rischel 2024 5 25 https://erischel.com/efr-0002/ efr-0002 /efr-0002/ Bidirectionality in graphical models Eigil Fjeldgren Rischel 2024 5 25 https://erischel.com/efr-0003/ efr-0003 /efr-0003/ Structural Causal Model Definition Let be a (finite) directed acyclic graph (DAG). Then a structural causal model (SCM) on consists of For each vertex , measurable spaces A family of independent random variables ---these are called the exogenous variables For each , a measurable function (note that if has no parents, the product is just a singleton) There is a significant existing literature (see Abstracting causal models, Reference , Reference . For a survey see Reference ) studying transformations between structural causal models. These have generally been called something like abstractions, the typical examples being the relationship between a high-level and a low-level model of the same system. However, the precise properties that we should ask for in such a transformation have turned out to be somewhat subtle. The most obvious condition to impose is to ask, for any possible variable we could intervene on, say , and any other variable (or collection of variables) , that the diagram of kernels commutes---in other words, given an intervention in the low-level model, the two possible distributions in the high-level model agree. The main problem with this condition is that it is simply too strict. We generally can't ask that every low-level intervention is well-modeled by the corresponding high-level intervention. Rather, each high-level intervention corresponds to some distribution of the corresponding low-level variables, and we must ask that a diagram of this form: commutes. For example, if the low-level variables are the velocities of all the gas molecules in a container, and the high-level variables are thermodynamic quantities like temperature and pressure, we can't expect that every set of velocities corresponding to a given temperature will lead to a given distribution of outcomes---rather, intervening on the temperature leads to some particular distribution of possible sets of velocities, and this leads to the observed distribution of outcomes. The key here is that control information flows in the opposite direction from measurement information. This phenomenon repeats in many examples. Eigil Fjeldgren Rischel 2024 5 25 https://erischel.com/efr-0004/ efr-0004 /efr-0004/ Bidirectionality for control Consider a discrete-time, continuous-space, controlled, nonlinear dynamical system with noise. By this, I mean some transition function along with a given "noise distribution" on . The intended meaning is that , where is a variable which can be controlled (depending on ,) and is a sequence of iid random variables distributed according to (for simplicity, we assume everything is time-invariant). Without any assumptions of linearity or the like on , it is hard to apply traditional control methods. Therefore we'd like to partition into a discrete set of components, and apply reinforcement learning algorithms to the resulting dynamics (or just straightforward dynamical programming, depending on the situation). Suppose we have such a partition , with some discrete index set. We want to build some sort of controlled dynamical system on (really a Markov Decision Problem). A priori, however, it seems the probability of being able to transition from to is highly dependent on where in we are. The trick to building an MDP is to choose , where is our current state and is the target region, in such a way that does not depend on ---then the distribution on the next state is just the distribution of , which can be determined ahead of time for each . is a section of ---this bidirectionality is completely analogous to what we saw for abstractions between graphical models. The datum is the backwards part of a lens . Actually, this raises a subtlety we swept under the rug above---it may not be possible to find satisfying for all pairs . Hence we need to restrict ourselves to those where it is possible---and because we're trying to abstract away , we need to consider those so that it's possible for all in our current component. This will bring us to consider dependent lenses. Eigil Fjeldgren Rischel 2024 6 5 https://erischel.com/efr-0009/ efr-0009 /efr-0009/ Composition with graded predicates Suppose is a stochastic map, and suppose is a graded predicate on (in particular, if ). Then we can define , a graded predicate on , by if there exists with 1-\epsilon _2]]> Every graded predicate on is the pullback of the predicate on along a unique stochastic map (which carries to the distribution which is with probability ). Thus a graded predicate is essentially a stochastic version of an ordinary predicate (which is a deterministic map to ). Under this equivalence, the operation is simply precomposition with . Let be two stochastic maps to the subobject classifier (i.e ). Defining logical operations on such objects is problematic essentially because of the lack of independence---if classify the probability that some lie in two different subsets , there is essentially no way to recover the probability of the intersection, or the union, since we can't see their correlation. We can consider a few different options---writing for the probability of truth (assuming that we're in a setting where ). is the probability of if they are maximally disjoint, i.e completely disregarded the possibility of overlap. Dually, is the probability of if they are maximally disjoint is the probability of if they are maximally overlapping, and dually is the probability of if they are minimally overlapping is the probability of if they are independent, and dually is the probability of if they are independent In the language of error credits, adding an error credit of amounts to taking with the "maximally disjoint" assumption (this gives the smallest possible probability of truth, i.e. it's the strongest assumption you can make). Eigil Fjeldgren Rischel 2024 7 4 https://erischel.com/efr-002Q/ efr-002Q /efr-002Q/ Definition Let be a weak 2-fibration of bicategories (in the sense of Buckley). Then the corresponding functor factors through if and only if each mapping functor is a discrete fibration. Eigil Fjeldgren Rischel202474Proof Let be a 2-fibration with this property. Note that is a full sub(tri)category, hence the only question is whether the fibers are categories---that is, whether they have discrete hom-categories. The mapping category in the fiber, given , is given by the fiber of over the identity---but by assumption this is a discrete category. Conversely, let be a bifunctor and consider its Grothendieck construction. By definition, a 2-cell is where , and is a 2-cell like this in but such a 2-cell exists if and only if this diagram commutes. Hence, given the base and the lift there is a unique choice of completing this lift, namely must be the composite in that triangle. This exactly proves that is a discrete fibration on hom-categories. (Note, we use the notation for the Grothendieck construction/category of elements, where Buckley uses ) Eigil Fjeldgren Rischel 2024 6 14 https://erischel.com/efr-000G/ efr-000G /efr-000G/ Constructing the triple category of controlled processes Triple categories are very complicated objects. And our life is further complicated by the fact that, in one direction, our triple category is not in general strict, in the sense that composition is only associative up to globular isomorphism (because the tensor product is only pseudoassociative, the composition of controlled processes inherits this defect). Hence, our life will be greatly simplified if we can approach the construction of this triple category by more conceptual means. Our strategy will essentially be the following: We will note that the notion of internal pseudocategory is classified by a limit sketch---that is, there is a bicategory (actually, a 1-category) and class of diagrams so that a pseudocategory internal in is precisely a pseudofunctor which carries all the diagrams in to homotopy limits We will prove that the double categorical construction preserves homotopy limits. Hence, by the preceding point, applying it to a pseudocategory internal to the bicategory of category actions will produce a pseudocategory internal to pseudo double categories---a triple category. Eigil Fjeldgren Rischel 2024 6 25 https://erischel.com/efr-001M/ efr-001M /efr-001M/ Limit sketch Definition A limit sketch consists of a (small) category equipped with a collection of cones (recall that is the "cone on ", the result of freely adjoining an initial object to ). If is a category, a model of the limit sketch is a functor so that each composite is a limit cone. A morphism of models is any natural transformation. In other words, the category of models is the full subcategory of the functor category spanned by the models. It is denoted (the collection of diagrams is left implicit). Given a bicategory or other higher category , a model is a pseudofunctor which carries every diagram to a homotopy limit cone. is again the full sub-bicategory of the bicategory of pseudofunctors spanned by the models. We will see that there are limit sketches classifying both internal categories (giving double categories) and symmetric pseudomonoids (giving symmetric monoidal categories), which will be important going forward. Eigil Fjeldgren Rischel 2024 6 25 https://erischel.com/efr-001O/ efr-001O /efr-001O/ Pseudocategories and categories of categories Remark We will want to say that the category of pseudo double categories is the category of internal pseudocategories---modeled as Segal objects---in the bicategory of categories. There are two issues with this statement as written. First of all, a priori, a weak Segal category is quite different from a pseudo double category. Not only is there not a canonical choice of composition given two horizontal maps , we cannot even (a priori) guarantee the existence of a choice of composition which has the right boundaries on the nose---only up to vertical isomorphism (so we can find a composite and isomorphisms ). On the other hand, a weak Segal category is also stronger than a pseudo double category, in the sense that every vertical isomorphism has a companion. We will see that the first defect can always be remedied after replacing our weak Segal category with an equivalent one. The issue of companions is a fundamental difference, but luckily all of the relevant double categories we want to consider have these companion pairs. The second question is somewhat subtler (but also less important). The category of internal pseudocategories, computed in any higher category, has a priori the same homotopy level as the input---just as the category of monoids is a 1-category (like ,) but the category of monoidal categories is a bicategory. This makes sense if we view categories as just a plain (generalized) algebraic theory, but we usually want to think of categories of categories as having some extra homotopy theory coming from the category structure---the category of categories in is a bicategory (actually a 2-category, i.e it's strict), not a plain category. The preceding construction does not account for this---the homotopy limits implicit in the phrase "a pseudocategory in pseudo double categories" are, for us, merely homotopy limits of plain categories, not accounting for the extra categorical structure in a double category. It is not clear what this means in general (and in any case, our output category will actually be strict in two directions,) but it is an important subtlety to remember. (A category viewed as an object merely of the 1-category of categories is sometimes called a strict category, but obviously we can't speak about "strict pseudo double categories") Eigil Fjeldgren Rischel 2024 6 25 https://erischel.com/efr-001N/ efr-001N /efr-001N/ Segal object Definition Consider the simplicial category . For every , the following square commutes: A simplicial object in a category is called a Segal object if each of these squares goes to a pullback square. Eigil Fjeldgren Rischel 2024 6 25 https://erischel.com/efr-001P/ efr-001P /efr-001P/ Lemma Let be a (strict) simplicial category (note that every simplicial category is equivalent to a strict one). Suppose further the face maps are both isofibrations. Then if is a Segal category in the strict sense, it is automatically a Segal category in the pseudo sense. Eigil Fjeldgren Rischel2024625Proof Since, up to isomorphism of categories, the higher are pullbacks of the two indicated face maps, and the projections in the diagram are compositions of pullback projections, and pullbacks and composition of isofibrations are again isofibrations, it follows that all of these squares are pullbacks of two isofibrations. Such a pullback is always also a homotopy pullback, which proves the first statement. Eigil Fjeldgren Rischel 2024 6 24 https://erischel.com/efr-0016/ efr-0016 /efr-0016/ Simplicial Category Definition The simplicial category is the category of finite, nonempty, totally ordered sets, and order-preserving functions, denoted . We denote by for the ordered set of elements---every object on is isomorphic to one of the form . A simplicial object in a category is a functor . Eigil Fjeldgren Rischel 2024 6 24 https://erischel.com/efr-0017/ efr-0017 /efr-0017/ Proposition Let be a category with finite limits. The internal nerve functor is fully faithful, and the essential image consists exactly of those simplicial objects where the maps given by evaluating at , exhibits as the iterated pullback Eigil Fjeldgren Rischel 2024 6 24 https://erischel.com/efr-0018/ efr-0018 /efr-0018/ Proposition A pseudo double category is equivalently a simplicial object in so that the diagrams are homotopy pullbacks. Eigil Fjeldgren Rischel 2024 6 26 https://erischel.com/efr-001Q/ efr-001Q /efr-001Q/ Commutativity of internalization Lemma Let be limit sketches, and let be a bicategory. Then where denotes the full subcategory of the functor bicategory spanned by those pseudofunctors so that, for each , is a model of , and for each is a model of . Eigil Fjeldgren Rischel2024626Proof Each isomorphism is an immediate consequence of the obvious fact that (pseudo)limits in model (bi)categories are computed pointwise, and the standard currying equivalence. Eigil Fjeldgren Rischel2024626Remark Note that if is a 2-category, so that we can speak of both pseudolimits and ordinary limits in , taking strict models does not generally commute with taking pseudo models, since it is not generally the case that pseudolimits commute with limits. Eigil Fjeldgren Rischel 2024 6 26 https://erischel.com/efr-001R/ efr-001R /efr-001R/ Remark A weak Segal category is similar to a virtual double category: we don't directly have a composition operator ---rather, we have a category of "compositions" each of which contains three faces in . Hence we have a choice of composites for every composable pair, although it is uniquely defined up to contractible choice. Eigil Fjeldgren Rischel 2024 6 26 https://erischel.com/efr-001S/ efr-001S /efr-001S/ Coherent nerve of pseudo double category Definition Let be a pseudo double category. Then the coherent nerve is the simplicial category defined as follows: The objects of are tuples , where are objects of , are horizontal morphisms, and is a globular isomorphism in The morphisms of are triples of 2-cells so that the two possible 2-cells agree Given four compatible objects of defining a 3-boundary (a tetrahedron), there is a unique objects of filling it if and only if the square of globular isomorphisms commute Given four compatible morphisms of , there is always a unique morphism in filling the boundary The face- and degeneracy functors are the obvious ones is -coskeletal, which defines the rest of the structure Eigil Fjeldgren Rischel 2024 6 26 https://erischel.com/efr-001T/ efr-001T /efr-001T/ Theorem A pseudo double functor is equivalent to a natural transformation If is a double category in which each vertical isomorphism has a companion, is a weak Segal category Every weak Segal category is equivalent to one of the form for a pseudo double category of this type. Let be a double category in which every vertical isomorphism has a companion. Then by reference here, Shulman stuff, the source and target functors are isofibrant. To prove is a weak Segal category, it therefore suffices to prove that is an equivalence of categories, and the analogous statement in level 3 (it follows for the higher levels by coskeletality). Now, let be a weak Segal. By reference, every simplicial category is equivalent to one which is Reedy fibrant. Note that being a Segal category is preserved by equivalence, so without loss of generality, assume is Reedy fibrant. This means the functor is an isofibration for each . First, let . This means is an isofibration. Since such fibrations are stable under pullback, this implies that the projection is again an isofibration, which means the composite is an isofibration. Since this pullback is a homotopy pullback, and is weak Segal, it is an equivalence, and hence a trivial equivalence. Choose a (strict!) section . By a completely analogous argument, choose a section of which is again a trivial fibration. Let our pseudo double category have as the underlying reflective graph, composition defined by the composite associator defined using the degeneracies, and unitor defined using the section of . Call this pseudo double category It is easy to see there is a functor , which is the identity on the first two levels. Since the higher levels are homotopy pullbacks of the lower levels, this implies it is an equivalence at each level, concluding the proof. Eigil Fjeldgren Rischel 2024 6 14 https://erischel.com/efr-000H/ efr-000H /efr-000H/ preserves homotopy limits Theorem The pseudofunctor , which carries a symmetric monoidal functor to the pseudo double category , preserves homotopy limits. Eigil Fjeldgren Rischel2024614Proof Note that homotopy limits on both sides are computed levelwise (using the observation that both are categories of models of (homotopy) limit sketches in the bicategory .) Since this immediately implies that preserves homotopy limits. It suffices to verify that also preserves them. But note that can be written as the comma object of the cospan and since comma objects are PIE limits, they commute with homotopy limits (pseudolimits). This concludes the proof. Note that (pseudo-) monoids equipped with a left module in a general bicategory are also classified by a limit sketch, and both the underlying object of the module and the map are in the image of this limit sketch. By instantiating this for being the bicategories of categories (or monoidal categories, etc), we recover alternative versions of Theorem for category actions that are not symmetric. We will mainly concern ourselves with the symmetric case, however. Eigil Fjeldgren Rischel 2024 6 14 https://erischel.com/efr-000I/ efr-000I /efr-000I/ Definition Let be a dynamical systems theory, let be the associated category of arenas. Recall that is the category of lenses. Consider the (strict) category internal to given by the following diagram: By Theorem , the image of this internal category under is a iwss triple category. We call this triple category the triple category of controlled processes, and denote it by , or if we want to disambiguate the choice of doctrine. (Note the difference between and ---the former has objects all morphisms of , and the morphisms between them given by isomorphisms, whereas the latter has objects only the isomorphisms in ) Eigil Fjeldgren Rischel 2024 6 14 https://erischel.com/efr-000J/ efr-000J /efr-000J/ Proposition The objects of are arenas. The three directions of morphism in are lenses, charts, and controlled processes. The lens-chart double category is isomorphically ---in particular, composition of both lenses and charts is strictly associative. The process-chart double category has 2-cells the chart reparametrizations. The process-lens double category has 2-cells the lens isoparametrizations. They both compose in the obvious way. Processes compose as morphisms of ---that is, this is associative up to a coherent choice of globular isoparametrization. Eigil Fjeldgren Rischel 2024 6 17 https://erischel.com/efr-000U/ efr-000U /efr-000U/ We can consider two strategies for constructing the "full" triple category of controlled processes: Give a construction of the double category of controlled processes and all lens reparametrizations, in a functorial way so that we can extend this to a triple category Prove that freely(?) adding companions, in a suitable sense, of each globular chart reparametrization, gives the right thing (this should amount to checking a composition rule) Eigil Fjeldgren Rischel 2024 6 17 https://erischel.com/efr-000V/ efr-000V /efr-000V/ Proposition Let be constructed by freely adding companions of every globular chart reparametrization. Then admits the following description: The objects are the objects of i.e tuples The vertical morphisms are chart reparametrizations The horizontal morphisms are lens reparametrizations is thin, and a square exists if and only if the underlying square in commutes. Moreover, the internal category structure , and the symmetric monoidal structure, both extend to so that we obtain a symmetric monoidal triple category Fix the notation of vs Eigil Fjeldgren Rischel2024617Proof It is apparent that the given data describes a double category which receives a functor from . It is also apparent that globular chart reparametrizations have companions in this double category, given by the equivalent globular lens reparametrization. Since, as developed in insert example here, we have seen that every lens reparametrization factors as a composite of companions of lens isoparametrizations and companions of globular chart reparametrizations... finish this. Eigil Fjeldgren Rischel 2024 6 20 https://erischel.com/efr-000X/ efr-000X /efr-000X/ proposition The functor has a terminal coalgebra, carried by with the deterministic structure map . Eigil Fjeldgren Rischel2024620Proof Observe that this diagram is a Kolmogorov limit in This gives a coalgebra structure on in the usual way. Given , we can build a cone by iteratively applying to the unique map : This cone induces a map . Moreover, for such a map to be a coalgebra map is for two maps into to agree. Since the tensor product preserves the cofiltered limit defining , this is an equation between two cones on the sequence , which is exactly the coinductive equation defining the cone above. Eigil Fjeldgren Rischel 2024 6 25 https://erischel.com/efr-001L/ efr-001L /efr-001L/ The Double Category of Stochastic Arenas Let be a Markov fibration. Since it is a fibration, we may form the ordinary double category of arenas . Note that there is a double functor . This is a double fibration: an internal category in the category of fibrations. The double category of stochastic arenas is obtained by stochastically completing this fibration in both directions---in other words, we form the completions and . We must check that this still gives a double category (it does), which we then transpose, stochastically complete, then transpose again (once again, there is something to check to see that the completion preserves the double categorical structure). The phrase "stochastic arenas" is a bit unfortunate, since the arenas themselves are no different from ordinary arenas (stochastic completion does not change the objects). Rather, it is the morphisms, the lenses and charts, that are stochastic. Eigil Fjeldgren Rischel 2024 7 4 https://erischel.com/efr-002S/ efr-002S /efr-002S/ Stochastic lens Definition Let be a stochastically complete Markov fibration. Then a stochastic lens , where , with , consists of: An object , equipped with deterministic maps A (non-deterministic) section of And a map over Up to the equivalence relation identifying two such tuples if there exists (non-deterministic) so that the relevant diagrams commute---that is, the projections to and commute with , , and where is the unique map induced by the stochastic completeness property (This could have been described as a coend, but since we don't find the coend calculus very useful for manipulating the set of stochastic lenses, we choose not to.) Given an optic, the forwards part is the composite map . Eigil Fjeldgren Rischel 2024 7 15 https://erischel.com/efr-0030/ efr-0030 /efr-0030/ The opposite of Recall that (perhaps by definition, perhaps as a trivial theorem, depending on our choice of definition), there is a faithful functor carrying a set to the vector space freely generated by , and a stochastic map to the map which takes the basis vector to the linear combination . The essential image of this functor is, of course, the finite-dimensional vector spaces. The morphisms in the image are the stochastic matrices, which can be characterized as those that are positive (they carry vectors with positive coordinates to other such vectors) and preserve the constant vector (note that neither of these properties are invariant under isomorphism of finite-dimensional vector space). One way to think about this is that it proves is equivalent to the category of finite-dimensional, ordered, pointed vector spaces, where we assume that the chosen point must be positive in the order. (To see the inclusion is essentially surjective, note that given a vector space pointed by , all those coordinates positive, it receives an isomorphism from the same spaces pointed by given by the diagonal matrix with entries ). Now, let denote the category of finite-dimensional ordered vector spaces. Then equipping a space with a positive point is exactly choosing an order-preserving linear map , so we have proven . Now we can compute where the last isomorphism is via transposition (note that transposing a positive matrix yields a positive matrix). This isomorphism merely describes the fact that stochastic maps can also be described as linear maps so that . This map interprets a vector as a function on , and carries it to the function on which finds the expected value if is distributed as . The normalization condition merely says a constant function always has that constant as its expectation. The self-duality of does not generalize to infinite-dimensional vector spaces, so once we move beyond finite state spaces, this story will get somewhat more complicated. However, the basic idea that, to carry the tools of coalgebraic modal logic into the stochastic case, the sort of "predicate" we should consider is actually a function on the state space, will remain the central idea. (Indeed, to consider the right notion of stochastic function between, for example, smooth manifolds, we will in any case want to use the tools of functional analysis to think of these in terms of transformation on integration operators, which is essentially this idea). Another way to justify this idea is to note that is the subobject classifier for . Hence if we want our notion of predicate to transform under stochastic maps, and include the logical predicates in the usual sense, we have to at least consider the stochastic maps , which of course amount to functions in this case. But a positive linear map on functions is determined by what it does to functions like this, so whether we work only with these or all functions is more or less a matter of taste. Instead of having a logic consisting of the operations of Boolean algebra, augmented with extra operations described by , we have the operations of a vector space, augmented by some extra modal operations. The Markov structure of or whichever category we work in, will give the vector space of functions an -algebra structure, but we should note that some equations fail to hold in general (dual to the fact that not all morphisms in the Markov category are homomorphisms for the comonoid structure). The Hennessy-Milner property is essentially the same in this situation, saying that our set of modally-expressible functions should separate points in the terminal coalgebra. Since we will have multiplication and linear combinations of functions, this entails under some mild topological conditions that it's dense in the set of functions on the terminal coalgebra. It is worth examining this analogy a bit further, in the simpler setup of coalgebraic modal logic. If is a space in some general sense, let us schematically write for the space of functions on , so that as above stochastic maps are identified with certain linear maps . Again, the Markov structure on the category of stochastic maps (whatever it is) amounts to the fact that carries the structure of an -algebra, and deterministic maps are those which are not algebra homomorphisms. Let now be a simple stochastic dynamical system which outputs a value of type at each step. The dual of this is where the tensor here is now the tensor product of vector spaces. Among algebras, the tensor product is the coproduct, so we may restrict the above map to form two modal operators, and ---the latter being simply the transition operator taking a function to the expected value of the next step, the former computing the expected value of the output given . But since the transition may be nondeterministic, these two restrictions do not determine it. From a logical point of view, the problem is that we can't express statements about the correlation between the output and the next step. As a simple counterexample, consider two system where . In the first, the output and the next state are independent, and both equal the current state with probability . In the other, the output and next state are always equal, and again are equal to the current state with probability These systems have the same marginals and , and hence the two modal operators above have the same behaviour, and hence every modally expressible function is the same. But in the former, the probability of seeing ab output sequence beginning when starting in state is proportional to , whereas in the latter it is proportional to (since you have to flip twice), so these systems have very different behaviour. Using § , we can deduce a version of Hennessy-Milner for stochastic transition systems (with no input and outputs in finite set ). Namely, fix a system , and let with and denote the function which carries to the expected value of when are distributed according to the system. Then is precisely the bilinear function dual to . Since is the terminal coalgebra, and the inverse limit of , we can infer that is the colimit of the sequence in some category of -algebras (we must think a little about topology to make this precise.) But this should imply that the union of the images of inside this algebra is dense. So certainly they separate points. But this image consists exactly of formulas given by functions on , applications of and sums and products. So these formulae are sufficiently expressive to separate points of the terminal coalgebra (which are, of course, exactly bisimulation classes). Eigil Fjeldgren Rischel 2024 11 19 Logic for systems misc Eigil Fjeldgren Rischel 2024 6 24 https://erischel.com/efr-001B/ efr-001B /efr-001B/ Stochastic Categorical Systems Theory (The Discrete Case) It will be instructive to work out the theory in the discrete case, which is to say, for the Kleisli category of the discrete (i.e finite-support) distribution monad on . Note that even for discrete-time discrete-space applications, this setup is not really adequate, as we lack the structure to construct terminal systems. Eigil Fjeldgren Rischel 2024 6 24 https://erischel.com/efr-001C/ efr-001C /efr-001C/ Definition Let carry a set to the set of finite-support probability distributions on . Recall that is a monad. Given two indexed sets an indexed stochastic lens is an element of the convex space where the internal hom, product and coproduct are taken in the category of convex spaces. Note that if each is identical, say , this is isomorphism to Recall also that an element of is equivalently a function i.e. a Kleisli map . An indexed stochastic chart is an element of the convex space Eigil Fjeldgren Rischel 2024 6 24 https://erischel.com/efr-001D/ efr-001D /efr-001D/ Proposition The set of indexed stochastic charts is just the set of commutative squares in of this form: (where the vertical maps are, by assumption, deterministic, but the horizontal ones aren't necessarily) Eigil Fjeldgren Rischel2024624Proof Note that an indexed stochastic chart is equivalently a family of stochastic charts for each . The analogous statement is true for commutative diagrams of this form (since coproducts in diagram categories are pointwise, and coproducts of sets are again coproducts in Kleisli categories). Hence it suffices to check the case . In this case, a commutative square amounts to a family of distributions on each of which induces the same distribution on . On the other hand, we are looking at the convex set where this exponential denotes the iterated Cartesian product in convex spaces. An element of this set clearly gives a commutative square, because for each , there is a square which deterministically chooses that , and uses the given family of distributions on depending on . This mapping is injective, since each map can be recovered as a conditional distribution, and the convex combination of s is merely the underlying distribution on . On the other hand, by choosing conditional distributions, it is also seen to be surjective, finishing the proof. Eigil Fjeldgren Rischel 2024 6 28 https://erischel.com/efr-001W/ efr-001W /efr-001W/ Quantitative logic for systems Eigil Fjeldgren Rischel 2024 6 28 https://erischel.com/efr-001V/ efr-001V /efr-001V/ Definition Let be a system with interface , and let be another arena. A -valued predicate on is a lift For discrete-time deterministic systems, a -valued predicate amounts to a mapping ---since the diagram implies the update can't depend on the extra input , and in the deterministic case, a lift of over is simply a choice of . (In the non-deterministic case, this is not true, of course). Let be a functor, and let be a coalgebra of it. Then we can describe a predicate of valued in as simply a function . Note that this makes a coalgebra of and as such it has a unique map to the terminal coalgebra (if it exists) of this functor . This means, given a predicate on that terminal coalgebra, by composing these two operations, we get a new predicate on . Thus, predicates on these cofree coalgebras are a sort of "modal operators"---for example, if is the set of booleans, and (so that coalgebras are just discrete dynamical systems), the cofree coalgebra on is the set of streams of booleans. Then we can take to be true if and only if each is true. Then the modal operator of is essentially "necessity"---the composite predicate is true for exactly those so that is true for every on the trajectory. Eigil Fjeldgren Rischel 2024 6 28 https://erischel.com/efr-001X/ efr-001X /efr-001X/ Definition Let be a lens, let be the terminal -system, and consider the composite . A -modal operator is a factorization of this system over , i.e some other -system with the same state space, so that the two composites agree. Let be some other system. It has a unique map to the terminal system. After composing with we can regard this as a map . We can then compose this with the unique map from to the terminal system, obtaining a new map This is the operation corresponding to . In good cases, there will be a monadic adjunction associated to the lens and so this operation will give an endomorphism on predicates for each choice of lift of the cofree predicate (it can of course lift as itself, which corresponds to the identity operation). Eigil Fjeldgren Rischel 2024 6 28 https://erischel.com/efr-001Y/ efr-001Y /efr-001Y/ When moving from deterministic, ordinary differential equations, to stochastic differential equations---or, from another point of approach, moving from discrete-time stochastic systems to continuous-time stochastic systems---we are forced to confront the problem that typical stochastic differential equations do not have differentiable solutions. A few questions immediately pose themselves when confronted with this statement: Since a solution of a stochastic differential equation is not merely a function but some sort of distribution of functions, what do you mean by them not being differentiable? Whatever the solutions are, how can they solve a differential equation if they aren't't differentiable? It will be instructive to consider Brownian motion. For each , let be an independent standard Gaussian. There are some issues with defining the integral since can't be guaranteed to be even measurable, but these can be sidestepped without too many problems. Then is Brownian motion, but with probability it is nowhere differentiable. The integral equation above describes the sense in which solves a stochastic differential equation saying it's derivative at each point is standard Gaussian distributed. Of course, since not all systems are integrable (this is not even true in the deterministic case,) there is not gonna be an integral operator lying around that we can use to define the solutions of our differential equations. Hence we must use a different solution concept. Recall that, if are differentiable functions, (integration by parts). Now if we choose to be zero except on a bounded interval, and let the endpoints to infinity, we get . The trick here is that the right-hand side is defined even if is not differentiable, and so it gives us a way of talking about "the derivative of ", at least for some purposes, even when it doesn't exist. Since two functions have the same integral after multiplication by every differentiable , if and only if they're equal almost everywhere, this is fairly robust---if is and satisfies some differential equation "weakly", it follows it must satisfy it on the nose as well, for example. Eigil Fjeldgren Rischel 2024 6 28 https://erischel.com/efr-001Z/ efr-001Z /efr-001Z/ Riesz Representation Theorem Let be a locally compact Hausdorff topological space, and let denote the vector space of complex-valued, compactly supported functions on . Recall that a linear functional is positive if it carries functions valued in the nonnegative reals to nonnegative real numbers. For every positive linear functional there exists a unique Radon measure on so that (conversely, it is clear that given a Radon measure, this equation defines a positive linear functional). Let be a positive linear map. Then dualizing gives a linear map . If we assume the initial map preserves the constant function, this will preserve probability measures, making it a sort of infinity-dimensional stochastic matrix. In particular, we get a composite map where the first map just includes the dirac measures (or, alternatively, takes to the "evaluate at " functional). If we further assume is continuous for the uniform convergence topology, will be continuous for the pointwise convergence topology. Since every probability measure is a limit of linear combinations of dirac measures in this topology, that means will be controlled by the above map, so we really have an injective correspondence between linear, continuous and unit-preserving maps and some subset of the Markov kernels . If we stare at this sequence of composites a bit, we realize that the linear operation corresponding to a kernel carries a function to the expected value function . Not all Markov kernels will have the property that this preserves continuity of (all measurable functions are Markov kernels, even)---but we would probably want to impose some notion of continuity on our kernels in any case. Instead of thinking about the classes of Markov kernels with this continuity-of-expectation property, we can simply use the space of continuous, unit-preserving linear maps as our class of stochastic maps Eigil Fjeldgren Rischel 2024 6 29 https://erischel.com/efr-0020/ efr-0020 /efr-0020/ SDEs are only diffusion and drift Naively, we would think that a stochastic differential equation on a manifold should be a stochastic section of the tangent bundle, (with the understanding that such a thing may not always have a solution). Indeed, we will see that such a thing does lead to a differential equation. But unlike the deterministic case, there is not a 1-1 correspondence---different sections may determine the same differential equation (in the sense of having the same solutions). The issue is that, as we have discussed, by necessity solutions of stochastic differential equations are defined by integration, in a certain sense. The solution concept we have in mind is a Markov process, where each derivative is sampled independently from its distribution. But by the central limit theorem, integrating a family of independent random variables will always give a normal distribution, no matter what those variables are. In particular, the only aspect of the assigned distribution of derivatives is its mean and its covariance matrix, which is the expectation of inside the tensor product when . (Another way of viewing the covariance is as an operator on the tensor product of the covector space , which carries a simple tensor of two covectors to the expected value of when is sampled according to the given distribution). Eigil Fjeldgren Rischel 2024 6 29 https://erischel.com/efr-0021/ efr-0021 /efr-0021/ In our study of discrete-time systems, we used the observation that the terminal system with a given interface can be characterized as the set of trajectories of that interface (eg § ). This allows us to develop an algebra of predicates on systems in the vein of coalgebraic modal logic. However, for continuous-time systems, the situation is much more complicated. In general, we can't expect an ordinary differential equation to have a unique solution for each initial value. There are two basic reasons for this: A simple system like experiences finite-time blowup, escaping to infinity inside a finite time horizon. In this example, the solutions have the form and so they clearly can't be continued past a certain horizon. Given some function we can't extend this to a function on all of The differential equation has both the solution and (and many others) The second issue can be solved by assuming our equations are Lipschitz (which the square root function is not). Since smooth functions are at least Lipschitz on compact domains (with Lipschitz constant bounded by the maximal derivative), there is some hope that we can get rid of this problem (but note that, when studying stochastic processes, there are reasons for wanting to study processes with jumps, complicating the situation). However, the first issue is more resilient. One approach is to restrict our attention to compact manifolds---this is enough to guarantee that every smooth differential equation has a (global) solution. The problem with this is that the spaces of trajectories---like the space of smooth functions ---will not even be a manifold, but some type of generalized smooth space. Hence we will not a priori be able to extend this class Eigil Fjeldgren Rischel 2024 6 29 https://erischel.com/efr-0022/ efr-0022 /efr-0022/ Theorem Consider the category of smooth microlinear spaces equipped with a map and a vector field . This category has a terminal object, given by the colimit taken over a suitable class of small objects. In other words, a point in the terminal system is given by some system and a point , up to an equivalence relation of bisimulation The mapping space , equipped with the evaluation at map and the flow is a subterminal object. A system admits a map to if and only if it is integrable in both directions Eigil Fjeldgren Rischel 2024 7 1 https://erischel.com/efr-002A/ efr-002A /efr-002A/ Review of coalgebraic modal logic Eigil Fjeldgren Rischel 2024 7 1 https://erischel.com/efr-002B/ efr-002B /efr-002B/ Weak limits Definition Let be a diagram. A cone is said to be a weak limit if, for any other cone , there exists a map (but this map is not necessarily unique). Observe that, if an actual limit exists, a cone is a weak limit if and only if the induced map admits a section. It follows that any functor which preserves the limit of some diagram also preserves the weak limits (in the sense that they are carried to another weak limit). Eigil Fjeldgren Rischel 2024 7 1 https://erischel.com/efr-002C/ efr-002C /efr-002C/ Proposition Let be a functor which preserves weak pullbacks. Then the assignment defines a functor (which just acts as on the objects). Eigil Fjeldgren Rischel202471Proof Recall that there is a functor , which carries a span to its image inside . Observe that two spans are identified under this mapping precisely if there exist span maps (not necessarily inverses) between them in both directions. It follows that, given two composable relations , the composite of and can be computed by taking the pullback , then taking its image inside . But since preserves weak limits, the map from to this pullback admits a section, and hence they go to the same relation. This proves that preserves composition (the claim about identities is straightforward). The basic idea is the following: suppose we have a system of formula we know how to interpret in coalgebras. This interpretation is a relation , given some coalgebra . Now, given an element , we can define its interpretation in simply using the lifting . And finally we can take the inverse image of the given subset of under to give a subset of For example, let so that a coalgebra is a simple labeled transition system. Then we get a modal operator for each . The meaning of is that and holds. If we let our basic language be Boolean logic, and our functor is not too bad (for example, if it's finitary), we get a result called the Hennessy-Milner property: two states satisfy the same formulae of the logic precisely if they are bisimilar. (It is not quite the same as saying formulas up to semantic equivalence are the same as bisimilarity classes---in the above example, we only get to ask about some finite prefix of the stream of outputs produced by . But of course two streams are equal if all the prefixes agree). Let denote the category of stone spaces, that is topological spaces which are compact, Hausdorff, and totally disconnected (meaning every connected component is a singleton). Then we have the following theorem, due to Stone Eigil Fjeldgren Rischel 2024 7 1 https://erischel.com/efr-002D/ efr-002D /efr-002D/ Stone duality Theorem If is a Stone space, the set of clopen subsets is a Boolean algebra. Moreover, the functor which carries a Stone space to its set of clopen subsets is an equivalence of categories. Now suppose our behaviour is given by an endofunctor . Then we immediately get a corresponding functor and the category of -coalgebras is dual to the category ot -algebras. Hence, giving a space the structure of a -coalgebra is equivalent to giving its set of clopen subsets a -algebra structure . This algebra structure can be interpreted as a (system of) modal operators---for example, if giving such a structure amounts to choosing maps and , corresponding to the operations "will produce an output that satisfies statement ?" and "will the next state after satisfy ?" We would like to move this framework to the setting of categorical systems theory. Of course, we are hampered by two immediate problems: our systems are not, in general, coalgebras of functors, nor are they in general sets. Eigil Fjeldgren Rischel 2024 7 15 https://erischel.com/efr-0030/ efr-0030 /efr-0030/ The opposite of Recall that (perhaps by definition, perhaps as a trivial theorem, depending on our choice of definition), there is a faithful functor carrying a set to the vector space freely generated by , and a stochastic map to the map which takes the basis vector to the linear combination . The essential image of this functor is, of course, the finite-dimensional vector spaces. The morphisms in the image are the stochastic matrices, which can be characterized as those that are positive (they carry vectors with positive coordinates to other such vectors) and preserve the constant vector (note that neither of these properties are invariant under isomorphism of finite-dimensional vector space). One way to think about this is that it proves is equivalent to the category of finite-dimensional, ordered, pointed vector spaces, where we assume that the chosen point must be positive in the order. (To see the inclusion is essentially surjective, note that given a vector space pointed by , all those coordinates positive, it receives an isomorphism from the same spaces pointed by given by the diagonal matrix with entries ). Now, let denote the category of finite-dimensional ordered vector spaces. Then equipping a space with a positive point is exactly choosing an order-preserving linear map , so we have proven . Now we can compute where the last isomorphism is via transposition (note that transposing a positive matrix yields a positive matrix). This isomorphism merely describes the fact that stochastic maps can also be described as linear maps so that . This map interprets a vector as a function on , and carries it to the function on which finds the expected value if is distributed as . The normalization condition merely says a constant function always has that constant as its expectation. The self-duality of does not generalize to infinite-dimensional vector spaces, so once we move beyond finite state spaces, this story will get somewhat more complicated. However, the basic idea that, to carry the tools of coalgebraic modal logic into the stochastic case, the sort of "predicate" we should consider is actually a function on the state space, will remain the central idea. (Indeed, to consider the right notion of stochastic function between, for example, smooth manifolds, we will in any case want to use the tools of functional analysis to think of these in terms of transformation on integration operators, which is essentially this idea). Another way to justify this idea is to note that is the subobject classifier for . Hence if we want our notion of predicate to transform under stochastic maps, and include the logical predicates in the usual sense, we have to at least consider the stochastic maps , which of course amount to functions in this case. But a positive linear map on functions is determined by what it does to functions like this, so whether we work only with these or all functions is more or less a matter of taste. Instead of having a logic consisting of the operations of Boolean algebra, augmented with extra operations described by , we have the operations of a vector space, augmented by some extra modal operations. The Markov structure of or whichever category we work in, will give the vector space of functions an -algebra structure, but we should note that some equations fail to hold in general (dual to the fact that not all morphisms in the Markov category are homomorphisms for the comonoid structure). The Hennessy-Milner property is essentially the same in this situation, saying that our set of modally-expressible functions should separate points in the terminal coalgebra. Since we will have multiplication and linear combinations of functions, this entails under some mild topological conditions that it's dense in the set of functions on the terminal coalgebra. It is worth examining this analogy a bit further, in the simpler setup of coalgebraic modal logic. If is a space in some general sense, let us schematically write for the space of functions on , so that as above stochastic maps are identified with certain linear maps . Again, the Markov structure on the category of stochastic maps (whatever it is) amounts to the fact that carries the structure of an -algebra, and deterministic maps are those which are not algebra homomorphisms. Let now be a simple stochastic dynamical system which outputs a value of type at each step. The dual of this is where the tensor here is now the tensor product of vector spaces. Among algebras, the tensor product is the coproduct, so we may restrict the above map to form two modal operators, and ---the latter being simply the transition operator taking a function to the expected value of the next step, the former computing the expected value of the output given . But since the transition may be nondeterministic, these two restrictions do not determine it. From a logical point of view, the problem is that we can't express statements about the correlation between the output and the next step. As a simple counterexample, consider two system where . In the first, the output and the next state are independent, and both equal the current state with probability . In the other, the output and next state are always equal, and again are equal to the current state with probability These systems have the same marginals and , and hence the two modal operators above have the same behaviour, and hence every modally expressible function is the same. But in the former, the probability of seeing ab output sequence beginning when starting in state is proportional to , whereas in the latter it is proportional to (since you have to flip twice), so these systems have very different behaviour. Using § , we can deduce a version of Hennessy-Milner for stochastic transition systems (with no input and outputs in finite set ). Namely, fix a system , and let with and denote the function which carries to the expected value of when are distributed according to the system. Then is precisely the bilinear function dual to . Since is the terminal coalgebra, and the inverse limit of , we can infer that is the colimit of the sequence in some category of -algebras (we must think a little about topology to make this precise.) But this should imply that the union of the images of inside this algebra is dense. So certainly they separate points. But this image consists exactly of formulas given by functions on , applications of and sums and products. So these formulae are sufficiently expressive to separate points of the terminal coalgebra (which are, of course, exactly bisimulation classes). Eigil Fjeldgren Rischel 2024 7 2 https://erischel.com/efr-002E/ efr-002E /efr-002E/ Discrete one-step modal operator Definition Consider the theory of discrete dynamical systems, and let be a system. Let be a predicate on , and let be a subobject of (that is, and and if then ). Then write for the predicate on which is true if and only if the following hold: and if then satisfies . In other words---the output is in , and if the input is further in then the next step will satisfy . Note that by taking this encodes a simple next-step modality. Observe also that, if are both finite, we can encode all the classical covering modality operators as (finite) conjunctions of and hence the logic built up of these modal operators and Boolean operations will enjoy the Hennessy-Milner property. Eigil Fjeldgren Rischel 2024 7 4 https://erischel.com/efr-002P/ efr-002P /efr-002P/ Logic for control Remark Let an MDP (or partially observable MDP) be given, and consider a statement like "given policy , the expected reward is ", or "the probability that the total reward is less than is ", and so on. In one sense, these are statements about policies---but, fixing a policy, they become statements about the states of the MDP (meaning: if the problem is initialized in state s_0, and we use policy , ...). In particular, they are statements about the behaviour of the state of the MDP, and so should be expressible using the modal logic. In particular, given a system if we have a good way of controlling the composite system ---that is, a modal formula providing that certain behavior on the interface will lead to a high reward---then we can translate this into a statement that certain behavior on the interface (namely, the same behaviour translated back) will lead to a high reward in the original system. Moreover, we're not restricted to statements about expected reward, but can encode all sorts of statements about the behavior, like a bound on the probability that some region will be crossed. Eigil Fjeldgren Rischel 2024 7 23 https://erischel.com/efr-003T/ efr-003T /efr-003T/ -coalgebras vs -coalgebras Both the endofunctor , and the endofunctor express the idea of a stochastic transition system which produces an output in the measurable space at each step. (We could also have considered the similar endofunctor on , using a discrete probability monad, and its Kleisli category). In fact, in one sense, these are equivalent---since is the Kleisli category of and on objects is simply defined as the product measurable space, the sets and are by definition equal to each other. The difference is in the category of coalgebras. A morphism between -coalgebras must be deterministic---it must assign to each a so that the obvious diagram commutes. In contrast, a morphism on -algebras is a morphism in ---it must assign to each a distribution on , so that the two implied distributions on agree. We can see the difference most clearly if we consider which properties are bisimulation invariant (i.e are properties of "behaviours"). We have seen that the terminal -coalgebra is , so a behaviour in this sense is simply a distribution on streams---the questions we can ask are what the probability of a certain sequence of outputs is. In contrast, of an element in a -coalgebra, we can ask a question like this: "What is the probability that, after 10 steps, we will be in a state which produces output with probability at least "? In a -coalgebra, we can ask "what is the probability that the output on the 11th step is ", or "what is the probability that, after 10 steps, the conditional probability of the next output being , conditional on the first 10 outputs, is at least ?"---but we cannot access the probabilities of an output given the future state directly, only the correlations that exist between the outputs. https://erischel.com/todo-tree/ todo-tree /todo-tree/ To do list Backlinks Related Contributions