Eigil Fjeldgren Rischel
https://erischel.com/index/
index
/index/
Eigil Fjeldgren Rischel
false
My name is Eigil Fjeldgren Rischel.
I'm studying for a PhD in Computer and Information Sciences at the Mathematically Structured Programming group at the University of Strathclyde.
My advisors are Neil Ghani and Radu Mardare.
I'm also a Research Consultant at the Tallinn University of Technology, in the Laboratory for Compositional Systems and Methods. I'm currently employed working on the Safeguarded AI project.
I mainly work on applications of category theory to probability and statistics.
This website is a "forest", made with the forester tool. It contains various notes I've made over the years, as well as some more structured blog posts.
Click on the headings below to expand, or on the bracketed link next to them to go to the relevant page.
Eigil Fjeldgren Rischel
https://erischel.com/efr-0032/
efr-0032
/efr-0032/
List of Publications
Radu Mardare
Neil Ghani
Eigil Fjeldgren Rischel
2025
9
17
https://erischel.com/mardare-ghani-rischel-2025/
mardare-ghani-rischel-2025
/mardare-ghani-rischel-2025/
Metric Equational Theories
Reference
10.4204/eptcs.428.11
Radu Mardare
Neil Ghani
Eigil Fjeldgren Rischel
2025
9
17
Abstract
This paper proposes appropriate sound and complete proof systems for algebraic structures over metric spaces by combining the development of Quantitative Equational Theories (QET) with the Enriched Lawvere Theories. We extend QETs to Metric Equational Theories (METs) where operations no longer have finite sets as arities (as in QETs and the general theory of universal algebras), but arities are now drawn from countable metric spaces. This extension is inspired by the theory of Enriched Lawvere Theories, which suggests that the arities of operations should be the lambda-presentable objects of the underlying lambda-accessible category. In this setting, the validity of terms in METs can no longer be guaranteed independently of the validity of equations, as is the case with QET. We solve this problem, and adapt the sound and complete proof system for QETs to these more general METs, taking advantage of the specific structure of metric spaces.
Eigil Fjeldgren Rischel
2025
6
2
https://erischel.com/rischel-cvxdual-2025/
rischel-cvxdual-2025
/rischel-cvxdual-2025/
Convex Duality Made Difficult
Reference
https://cgi.cse.unsw.edu.au/~eptcs/paper.cgi?ACT2025:28
Eigil Fjeldgren Rischel
2024
4
30
https://erischel.com/lcc-001Z/
lcc-001Z
/lcc-001Z/
Convex Duality made Difficult
Eigil Fjeldgren Rischel
2024
4
30
Introduction
The study of convex functions - in particular, of their optimization (really minimization) is one of the most important fields of applied mathematics. Convexity seems to be one of those incredibly well-chosen hypotheses which is just specific enough to admit a wealth of theorems, just general enough to produce a nontrivial theory (and a large amount of important examples).
Convex optimization, possibly because it has an "analytical" rather than "algebraic" feel, has not been very thoroughly studied by applied category theorists. The one notable exception is Reference , which studies the decomposition of optimization problems by categorical means. This paper takes a different approach, attempting to define a category with optimization problems as the objects, and to derive theorems about optimization by categorical means.
Eigil Fjeldgren Rischel
2024
4
30
Convex optimization
Eigil Fjeldgren Rischel
2024
4
22
https://erischel.com/lcc-001G/
lcc-001G
/lcc-001G/
Standard form convex optimization problem
Definition
A convex optimization problem in standard form consists of
A convex function
A list of convex functions
A list of affine functions
The problem then is to find which minimizes subject to the constraints
Eigil Fjeldgren Rischel
2024
5
17
https://erischel.com/lcc-002K/
lcc-002K
/lcc-002K/
Lagrangian of an optimization problem
Definition
Let be a standard-form convex optimization problem, as in Definition . Then the Lagrangian of this problem is the function defined by
Recall that denotes the nonnegative reals.
Observe that is if satisfies the constraints of the problem, and otherwise. Hence we can think of this minimization problem as playing a zero-sum game: we choose , our adversary chooses , and our loss function is .
It is natural to ask about the existence of Nash equilibria in this game - observe that the existence of an equilibrium means that . This is of great utility in solving the original problem.
The dual problem is the problem of maximizing the function . This is always a concave problem.
In the world of convex optimization, two problems whose constraints carve out the same subset of (and where the function to optimize is the same) would be called equivalent. But they can clearly not be regarded as isomorphic, because the choice of constraint functions makes an important difference to the theory of optimization (for example, it can lead to different dual problems). Here we take the viewpoint that the Lagrangian is really the fundamental object in convex optimization - by passing to a suitable category of Lagrangians, we can make the dual problem into an actual self-duality on this category.
Eigil Fjeldgren Rischel
2024
4
30
Convex spaces
Eigil Fjeldgren Rischel
2024
4
30
https://erischel.com/lcc-0020/
lcc-0020
/lcc-0020/
Convex Space
Definition
The category of convex spaces is the category of algebras for the monad of discrete finite-support distributions. The morphisms are called -homomorphisms or homomorphisms of convex spaces.
So as not to multiply notation unnecessarily, we simply denote the category of convex spaces by , using the usual notation for the Eilenberg-Moore category.
Eigil Fjeldgren Rischel
2024
5
20
https://erischel.com/lcc-002N/
lcc-002N
/lcc-002N/
Definition
A function between vector spaces is called affine if it preserves those linear combinations where
Eigil Fjeldgren Rischel
2024
5
20
https://erischel.com/lcc-002M/
lcc-002M
/lcc-002M/
Definition
Let be a convex space. A convex function on is a function so that
A concave function is a function so that is convex (in other words, satisfies the opposite inequality).
The term "convex function" in this sense clashes with the usual practice of naming structure-preserving functions after the structure they preserve (since convex functions do not preserve the convex structure). Unfortunately this usage is far too established to alter. (Convex functions are called convex because they are exactly those functions where the area above their graph is a convex subset of . Although there appears to be no particular reason why the terms convex and concave should not be interchanged, other than convention).
Eigil Fjeldgren Rischel
2024
5
19
https://erischel.com/lcc-002R/
lcc-002R
/lcc-002R/
Jensen's Inequality
The inequality
which holds whenever is convex, is called Jensen's inequality.
Sometimes this name is used for a stronger version of this inequality,
like the claim that if is a random variable valued in the domain of . These generally follow just from convexity of .
Eigil Fjeldgren Rischel
2024
5
19
https://erischel.com/lcc-002Q/
lcc-002Q
/lcc-002Q/
There is an natural way to extend the convex structure of to both and , by the convention that any nontrivial convex combination involving an infinity is equal to that infinity. This also gives the adjectives convex and concave a meaning when applied to functions . For example, a function is convex if and only if the subset where it's finite is a convex subset of , and it's a convex function in the ordinary sense on this set.
This doesn't work for the extended real line , since there is no sensible interpretation of . We will inescapably meet some functions which take value in the full extended reals, but where we still wish to speak of their convexity (or concavity).
Hence we adopt the convention that a function is convex if it obeys Jensen's inequality whenever it makes sense, i.e whenever we do not have or vice versa.
Eigil Fjeldgren Rischel
2024
5
20
https://erischel.com/lcc-002O/
lcc-002O
/lcc-002O/
Proposition
A function between vector spaces is affine if and only if it is a -homomorphism.
Eigil Fjeldgren Rischel2024520Proof
It's clear that an affine function is a -homomorphism.
Suppose is a -homomorphism. Note it suffices to prove preserves binary affine combinations (for not necessarily in ).
If , we are done by assumption. Otherwise suppose 1]]> (if not, replace it by by symmetry). Then
This is a convex combination, so
Rearranging, we find
as desired.
Justified by Proposition , we will appropriate the term affine to refer to -homomorphisms, even between convex spaces which are not vector spaces. There is generally no chance of confusion, but it's worth emphasizing that the use of this term does not entail that the domain is closed under arbitrary affine combinations, for example.
Convex spaces admit both a Cartesian product (given by the product of the underlying sets equipped with pointwise operations) and a tensor product, which (co)represents "bihomomorphisms". This is analogous to the situation for vector spaces. Unlike vector spaces, however, since all constant maps are homomorphisms, the projections are bihomomorphisms, which induces a map . Thus homomorphisms are a subset of bihomomorphisms.
Since we are generally dealing with convex or concave functions, which can't freely be extended to the tensor product, we will work with the Cartesian product in this paper. But it's very possible that most of our constructions would work also with the tensor product, and maybe there is some situation where the extra generality is necessary.
Eigil Fjeldgren Rischel
2024
5
21
https://erischel.com/lcc-002S/
lcc-002S
/lcc-002S/
Simplex
Definition
The free convex space on a finite set of elements is called the -simplex and denoted (the reason for the apparent mismatch of numbering is that the -simplex is -dimensional). Note that an element of is a tuple so that and .
In particular, .
Eigil Fjeldgren Rischel
2024
5
21
https://erischel.com/lcc-002U/
lcc-002U
/lcc-002U/
Topological convex space
Definition
A topological convex space is a convex space equipped with a topology so that any affine map is continuous (when is given the subspace topology).
Eigil Fjeldgren Rischel
2024
4
30
The Category of Minmax problems
Eigil Fjeldgren Rischel
2024
4
22
https://erischel.com/lcc-001C/
lcc-001C
/lcc-001C/
Minmax problem
Definition
A minmax problem is a triple where are convex spaces (that is, algebras for the discrete distribution monad---more topological assumptions may be necessary here), and is a function which is
Pointwise convex in ---for each , given ,
Pointwise concave in ---for each , given ,
A morphism of minmax problems is a pair of functions and so that
We will see that various constructions on this category, which are natural and well-behaved from the point of view of category theory, capture relevant constructions from the theory of convex optimization.
is bifibred over , and the Cartesian and coCartesian lifts capture the operations of minimizing over the primal variables or maximizing over the dual variables
The property of strong duality amounts to the claim that a particular diagram has the local Beck-Chevalley property
Relatedly, the existence of a Nash equilibrium for the game corresponding to amounts to the existence of a certain morphism. The fact that this implies strong duality can be derived by purely categorical means.
Eigil Fjeldgren Rischel
2024
5
10
https://erischel.com/lcc-002H/
lcc-002H
/lcc-002H/
Proposition
Let be a minmax problem.
Suppose is a convex subspace of a vector space , and is affine for each .
Then there exists functions , so that .
Observe that minmax problems affine in are thus very similar to standard-form convex optimization problems, the main difference being that the set of allowed points in may be constrained in some other way than by requiring certain coordinates to be nonnegative.
On the other hand, if is thus constrained, the proposition doesn't actually imply that are convex! The easiest way to see this is by considering for some nonzero . Then we have , and clearly we can choose this decomposition in such a way that these functions are not convex.
However, if contains the positive cone for example, we do have both and all the coordinates of convex.
Eigil Fjeldgren Rischel
2024
4
22
https://erischel.com/lcc-001D/
lcc-001D
/lcc-001D/
Primal and dual optimization problems
Definition
Let be a minimax problem.
The primal optimization problem associated to is the function
(the problem being to minimize this function).
The dual optimization problem is the function
Eigil Fjeldgren Rischel
2024
4
23
https://erischel.com/lcc-001J/
lcc-001J
/lcc-001J/
Dual minmax problem
Definition
Let be a minmax problem. Then let denote the dual problem given by .
If is a morphism of minmax problems, then is again a morphism in the other direction. This assignment makes into a self-inverse functor on the category of minmax problems
We will often utilize this duality to abbreviate proofs, proving something, for example, for the forwards direction and arguing "by duality" that it holds for the backwards direction as well.
Eigil Fjeldgren Rischel
2024
5
1
https://erischel.com/lcc-0027/
lcc-0027
/lcc-0027/
Backwards and forwards morphisms
Definition
Let a morphism in be called forwards if is an isomorphism, and backwards if is an isomorphism.
Let denote the set of forwards morphisms, the set of backwards. Then clearly form an orthogonal factorization system - in fact, both and do.
Note that consists exactly of the local equivalences for the inclusion of , so that the localization of can be formed as the terminal forwards map from it, which is clearly (of course, this is not surprising).
We will say a morphism in is forwards, respectively backwards, if it is so considered as a morphism in , and reuse the notation for these subclasses of morphism.
Eigil Fjeldgren Rischel
2024
5
7
https://erischel.com/lcc-002A/
lcc-002A
/lcc-002A/
Lemma
Let be convex spaces and let be a convex subspace. Let be a convex function.
Then is again convex.
Eigil Fjeldgren Rischel202457Proof
Let be given, and consider:
Since if then , we have that this is less than:
because in the latter we are taking the infimum over a smaller set of 's
Applying convexity, we get
This is precisely the desired inequality.
Eigil Fjeldgren Rischel
2024
4
30
https://erischel.com/lcc-0023/
lcc-0023
/lcc-0023/
Proposition
The forgetful functor is a bifibration. Moreover, we have the following description of the (co)Cartesian morphisms over backwards and forwards maps.
A forwards morphism is Cartesian if and only if for all
A forwards morphism is coCartesian if and only if
A backwards morphism is Cartesian if and only if
A backwards morphism is coCartesian if and only if
Eigil Fjeldgren Rischel2024430Proof
Note that it suffices to provide Cartesian and coCartesian lifts for backwards and forwards morphisms (Definition ), since such lifts compose. Hence it suffices to verify that the given descriptions are correct, since clearly they suffice to compute a (co)Cartesian lift over any such morphism.
Note also that, since the forgetful functor is faithful, to verify a morphism is (co)Cartesian, it suffices to prove that any factorization in the base lifts - uniqueness is automatic.
Thus let be so that . Note that composition of a -homomorphism with a convex function is again convex, so this is indeed an object of
Now let be some morphism so that we have the factorization in . The goal is now to prove is a homomorphism. This is the inequality
which holds by assumption
Let be as above, but suppose . First, observe that by lcc-002A, this function is in fact convex in as desired.
Let be given, and now suppose we have a factorization in the base. We must prove that
but since this amounts to the equation
which is again true by assumption.
Now the case for backwards morphisms simply follows by duality.
What's "really" going on here is that is a two-sided fibration, the result of taking the functor carrying a pair to the poset of minmax problems (in the opposite order), with morphisms acting by precomposition, and applying the Grothendieck construction "contravariantly in the first variable and covariantly in the second variable". (And then observing that the precomposition action has left/right adjoints given by /, to make this into a bifibration). But the theory of two-sided fibrations is quite complicated in general, and we will not go into it here.
Note also that this functor is quite close to displaying as topological. If we remove the restriction that minmax problems be convex/concave, we can construct the universal lifts required using a similar supremum formula. The problem is that the supremum of a general set of concave functions is not automatically concave (however, the supremum taken over a convex set, in a suitable sense, is).
Eigil Fjeldgren Rischel
2024
4
25
https://erischel.com/lcc-001R/
lcc-001R
/lcc-001R/
and
Definition
Let be the category where objects are pairs consisting of a convex space and a convex function, and where morphisms are affine maps so that .
Let be the category where objects are pairs consisting of a convex space and a concave function, and where morphisms are affine maps so that .
Eigil Fjeldgren Rischel
2024
4
25
https://erischel.com/lcc-001S/
lcc-001S
/lcc-001S/
Proposition
The assignment
defines a functor
Similarly, defines a functor . (The reason for this idiosyncratic way of writing a contravariant functor will become apparent in a minute)
The assignment defines a functor (identity on morphisms) , and vice versa. Then
The assignment defines a fully faithful functor ,
whose essential image consists of those tuples where is singleton.
Analogously, defines a fully faithful functor
We will abuse notation and identify and with their images under these inclusions - thus, for example, will be regarded as an object of .
is right adjoint to the inclusion of , and (viewed as a functor ) is left adjoint to the inclusion of
Using these identifications, we have
Note that if is a morphism of , the two meanings of the notation agree, and the same is true of .
Note also that the reflexive subcategory is the local subcategory with respect to the forwards morphisms - a morphism is forward if and only if is an isomorphism (by definition), and the unit is the terminal forwards morphism with domain . A dual statement holds for (it is the colocalization with respect to the class of backwards morphisms).
Eigil Fjeldgren Rischel
2024
4
23
https://erischel.com/lcc-001N/
lcc-001N
/lcc-001N/
Monoidal structure on minmax problems
Definition
There is a monoidal structure on minmax problems, given by
The unit here is .
A state is a point so that for all .
More interesting is asking for a state of .
This is a pair so that the inequality
holds for all
Note that for all , this is the minmax inequality (or "weak duality").
Thus a choice of giving a state gives equality in that inequation---it is a solution of the minmax game. In other words, has a state if and only if strong duality holds for , and the state is given by an optimal and dual optimal pair in that case.
(By duality, and since , such an object has a state if and only if it has a costate)
Eigil Fjeldgren Rischel
2024
5
8
https://erischel.com/lcc-002D/
lcc-002D
/lcc-002D/
Proposition
The forgetful functor is a monoidal fibration, in the sense of Reference , (see also Reference ). It is also a monoidal opfibration - in other words, both the classes of Cartesian and coCartesian maps are stable under tensor product.
In the case of a Cartesian base, a monoidal fibration (like the one we have here) is equivalent to a fibration with a monoidal structure on each fiber, compatible with the reindexing in a certain way. Our base is not Cartesian, but does seem to come from a monoidal structure on each fiber, given by addition of s. The point is that, given , there is a canonical way to obtain an on , given by using a Cartesian lift of and a coCartesian lift of . This suggests there should be a useful theory of monoidal two-sided fibrations, but this notion does not appear to have been studied before.
Eigil Fjeldgren Rischel
2024
5
19
https://erischel.com/lcc-002P/
lcc-002P
/lcc-002P/
Proposition
The localization (resp. colocalization) () is monoidal, in the sense that the class local equivalences is stable under tensor products. The thus induced monoidal structure on is given by (and the same for ). In particular the (co)localization functor is strong monoidal.
Eigil Fjeldgren Rischel
2024
4
30
Strong duality
Eigil Fjeldgren Rischel2024426https://erischel.com/lcc-001W/lcc-001W/lcc-001W/weak dualityProposition
Let be a minmax problem.
Then
where we abuse notation by identifying a minmax problem with the number
Eigil Fjeldgren Rischel2024426Proof
The equations are clearly true by definition.
Note that the inequality is equivalent to the existence of a morphism
We have canonical morphisms
Since is a right adjoint to the inclusion of , the above composite induces a map
Since is a left adjoint to the inclusion of , that map induces a map
as desired.
Eigil Fjeldgren Rischel202459https://erischel.com/lcc-002F/lcc-002F/lcc-002F/Strong dualityDefinition
Let be a minmax problem. By Proposition , there is a morphism
. We say satisfies strong duality if it is an isomorphism. (Note that this is really just an inequality of real numbers, which must be an equality).
Eigil Fjeldgren Rischel2024426https://erischel.com/lcc-001X/lcc-001X/lcc-001X/Proposition
Let be a minmax problem.
Suppose there exists .
Then strong duality holds, i.e gives a morphism
Here we use the isomorphisms and vice versa, as well as strong monoidality of and .
The existence of that morphism means that
which is the other direction of the morphism we wanted.
If a minmax problem is a zero-sum game, a point is a choice of Nash equilibrium for this game.
Eigil Fjeldgren Rischel202456https://erischel.com/lcc-0029/lcc-0029/lcc-0029/Proposition
Let be a minmax problem.
Then there is a canonical commutative diagram
in which is a pullback.
obeys strong duality if and only if this square has the local Beck-Chevalley condition for , in the sense that the canonical map is an isomorphism.
Eigil Fjeldgren Rischel202456Proof
Recall that .
Hence the claim is just that which is precisely strong duality.
(For more on the Beck-Chevalley condition, see Reference or nlab: Beck-Chevalley Condition)Eigil Fjeldgren Rischel2024518https://erischel.com/lcc-002L/lcc-002L/lcc-002L/Slater's Constraint QualificationProposition
Consider an optimization problem in standard form:
Minimize
Subject to
And
With each convex and each affine
Now assume that is such that are all affine (possibly , and none of the are affine), and suppose there exists so that for all l]]>, and for all other , for all . Then strong duality holds for this optimization problem, and the dual optimal value is attained.
Note: This is usually stated for a function defined on an arbitrary convex subset of . In this case we must further ask that is in the relative interior of this domain.
Eigil Fjeldgren Rischel2024518Proof
(Proof adapted from )
For simplicity, we will assume , i.e we will not assume any of the are affine, and for all . (This is the standard form of Slater's constraint qualification).
Let be a matrix and vector so that . Then assume without loss of generality that has full rank. (Suppose is in the span of the other s. Then if the affine constraints are feasible at all, the equation corresponding to must be a consequence of the others. Hence we can delete without altering the primal optimal value, and given a dual optimal value for the problem with deleted, just set .)
Let
Observe that the optimal value is .
Let . It's not hard to see that these sets are disjoint and convex, and hence there exists a hyperplane separating them.
In other words, there exists (not all ) so that
By the first inequality, we must have (or the right-hand side would be unbounded below on , which is impossible). Similarly we have .
The latter of the two statements is equivalent to the statement that when , which simply means . Combining this with the first statement,
we get for all
First assume . Then we can divide out and get , which proves that is a dual optimal value and that strong duality holds.
If , we have for all
Inserting , we find
and since we must have .
This then implies for all . But since this is an affine function, this can only be true if it's constantly zero. Since is nonzero and has full rank, this is impossible.
Eigil Fjeldgren Rischel2024422https://erischel.com/lcc-001E/lcc-001E/lcc-001E/Minimax theoremTheorem
Let . If are both convex, compact subspaces of finite-dimensional vector spaces, and is continuous, then strong duality holds for , and moreover an equilibrium exists.
This theorem can be derived from the Kakutani fixpoint theorem in a very similar way to the usual proof of Nash's theorem about general, non-zerosum games - although note that it is not a special case, since and may not be simplices, and the payoff function here is merely convex, not necessarily affine as it is for a game-theoretic game.
However, we will give a different proof, which uses the structure of in a more direct way. Essentially, we will use compactness to reduce to the case of simplexes, then use an inductive argument to reduce to the case where which can be shown by a direct topological argument. The inductive step is a fiber sequence argument, where we use the characterization of strong duality in terms of the Beck-Chevalley property, Proposition .
Eigil Fjeldgren Rischel2024521https://erischel.com/lcc-002T/lcc-002T/lcc-002T/Solvable pairDefinition
Let be topological convex spaces. We say the pair is a solvable pair if, for any continuous minmax problem , strong duality holds.
Eigil Fjeldgren Rischel2024521https://erischel.com/lcc-002V/lcc-002V/lcc-002V/Proposition
The pair (in other words, ) is solvable.
Eigil Fjeldgren Rischel2024521Proof
Let be a continuous minmax problem. Suppose strong duality does not hold. Then by adding a constant to , we can arrange that
Consider the set 0\}]]>. Since we must have 0]]> for each , the first projection must be surjective. Since each fiber is convex, and hence connected, and the projection is open, is connected. As an open connected subset of a convex space, it is path connected. Hence there exists some path where and . In other words (picturing the square with the first coordinate horizontal), there exists a path from the left to the right side of the cube so that 0]]> everywhere on the path. Dually, there also exists a path from top to bottom so that is strictly negative everywhere on that path. But they must intersect somewhere, and this is a contradiction. Hence must have a state or a costate, finishing the proof.
Eigil Fjeldgren Rischel2024521https://erischel.com/lcc-002W/lcc-002W/lcc-002W/Proposition
Let be a continuous, affine map between topological convex spaces. Let be another topological convex space, and suppose
is solvable.
For every , is solvable, where is the fiber.
Then also is solvable.
Eigil Fjeldgren Rischel2024521Proof
Recall that being solvable means the following square has the Beck-Chevalley condition for continuous :
Now we can factor this as follows:
Note that the right-hand square here has the Beck-Chevalley condition by assumption. So it suffices to show the left-hand square does. For a given , this means showing that these two functions on are the same
But this equation, for some given , is exactly strong duality in the restriction of to , which must hold because this is a solvable pair by assumption.
Eigil Fjeldgren Rischel2024521Corollary
If is solvable, so is for each , because the map which picks out the first coordinate has fibers isomorphic to so we can proceed by induction (the case being trivial.)
Eigil Fjeldgren Rischel2024521https://erischel.com/lcc-002X/lcc-002X/lcc-002X/Lemma
Let be a compact topological convex space. Suppose is solvable for all . Then is solvable for all topological convex spaces .
Eigil Fjeldgren Rischel2024521Proof
Suppose for contradiction does not have strong duality, and assume without loss of generality that . For each let consist of those so that . Given some finite family consider the induced map and apply solvability to the problem - this implies in particular that . This means the family has the finite intersection property, so by compactness it has nonempty intersection. But then an element of the intersection must satisfy which is a contradiction.
Eigil Fjeldgren Rischel2024521https://erischel.com/lcc-002Y/lcc-002Y/lcc-002Y/Corollary
If is compact, is solvable for any .
Eigil Fjeldgren Rischel2024521Proof
By Proposition and its corollary, applied to Proposition , we have solvable for all . Since is compact, this means is solvable for all , by Lemma . Now by duality we have solvable for all which means is solvable, and by using Lemma again, we're done.
Eigil Fjeldgren Rischel2024430
By Corollary , the pair is solvable, and strong duality holds. Since are both compact, there must exist attaining the infimum and the supremum . These form an equilibrium.
It is interesting to note the use of compactness here. Recall that topological compactness is closely connected with the property, also called compactness, of preserving filtered colimits (this property, instantiated in , is not actually the same thing as topological compactness). Our use of compactness here, to derive from the existence of a state in the "finitary" subproblems the existence of a state in the entire problem, does not have this form (nor is it even the case that is the colimit of its subsimplices), but it's possible that the proof could be rewritten to make this step more categorical.
The idea of proceeding by induction on was inspired by Reference , although our proof is rather different - they are only looking at affine games, and hence their induction step is completely different (and they have no need for the complicated base case that we do), and since we are not merely interested in games on simplices, we need an additional compactness argument.
We can use the minimax theorem to derive other statements of interest about convex optimizationEigil Fjeldgren Rischel2024522https://erischel.com/lcc-002Z/lcc-002Z/lcc-002Z/The separating hyperplane theorem (compact case)Theorem
Let be disjoint, compact, convex subspaces. Then there exists and so that whenever .
Eigil Fjeldgren Rischel2024522Proof
Consider the minmax problem
Since the closed unit ball is compact, by the minimax theorem there exists an equilibrium , which then satisfies
By disjointness, must be nonzero, so with a suitable choice of we can clearly make the left-hand item strictly positive. Hence 0]]>. Now there must exist some so that 0]]>.
By the equilibrium property, we see that must minimize on , and analogously must maximize on . Hence for all , we have
which concludes the proof.
Eigil Fjeldgren Rischel2024522https://erischel.com/lcc-0030/lcc-0030/lcc-0030/The separating hyperplane theorem (general case)Theorem
Let be disjoint convex subsets. Then there exists so that for all .
Eigil Fjeldgren Rischel2024522Proof
Let be two sequences of sets with the following properties:
For each , are disjoint.
For each ,
Each of the are compact and convex
These can be constructed for example by taking the intersection of and with the boxes to obtain compact, convex, disjoint subsets which exhaust and .
Now apply Theorem to obtain a sequence of so that is negative on and positive on . By compactness of the unit ball, this sequence has a point of density . Now for every pair , we can find some so that is within an arbitrary of and the same is true for , and so that . But then is within of which is positive, so that .
Now for each , has a maximizer on and a minimizer on . Hence, by an argument analogous to the proof of Theorem , there is a nonempty closed interval so that, if we have on and on . But since the sets are increasing this sequence of intervals must be decreasing, and hence the intersection must be nonempty - and then any in this intersection will make nonpositive on , nonnegative on , as desired.
Eigil Fjeldgren Rischel
2024
4
30
The Legendre Transform
Eigil Fjeldgren Rischel
2024
4
25
https://erischel.com/lcc-001P/
lcc-001P
/lcc-001P/
Convex conjugate
Definition
Let be a (real) vector space, and be a function (not necessarily linear).
Then the convex conjugate is defined by
The convex conjugate is also called the Legendre transform or the Fenchel-Legendre transform. It is intimately related to convex duality. We will prove the following fundamental property of the convex conjugate using the categorical language of minmax problems, and along the way we will see the role that convex duality plays. Note that our invocation of the term "strong duality" here is somewhat more complicated than strictly necessary - normally one would merely invoke the separating hyperplane theorem directly.
Eigil Fjeldgren Rischel
2024
4
25
https://erischel.com/lcc-001Q/
lcc-001Q
/lcc-001Q/
Proposition
Let be convex, so that is a minmax problem.
Then we can form the modified minmax problem - note that, up to a sign change in the domain, this amounts to adding the constraint .
Then
Note that the two uses of the asterisk in this equation conflict.
We have both the reversed optimization problem given by flipping the variables,
and the convex conjugate function . It may be good to alter this notation to resolve the conflict.
Eigil Fjeldgren Rischel
2024
4
25
https://erischel.com/lcc-001T/
lcc-001T
/lcc-001T/
Proposition
Given a minmax problem where is a finite-dimensional real vector space, let
Note that when and otherwise.
Thus this amounts to adding a constraint that .
Analogously, define
Then the Legendre transform (viewing both and as minmax problems using the inclusion )
Eigil Fjeldgren Rischel
2024
4
25
https://erischel.com/lcc-001U/
lcc-001U
/lcc-001U/
Proposition
Let be a continuous convex function defined on a vector space.
Then there is strong duality in the minmax problem
Eigil Fjeldgren Rischel2024425Proof
Observe that is a closed convex set.
Hence there is a hyperplane through so that the entire set is in one half-space.
This means a nontrivial affine equation which is satisfied whenever , and where .
Clearly for some .
If we have for all , which impossible.
So by normalizing let's set .
This means .
Recall that the minmax problem is given by .
Strong duality means .
We always have the inequality , so it suffices to identify an so that
Clearly, for our , we have , since the supremum is unless .
On the other hand, taking , we have for all by construction, finishing the proof.
Eigil Fjeldgren Rischel
2024
6
4
https://erischel.com/lcc-003C/
lcc-003C
/lcc-003C/
Lemma
Given a commutative square:
in with the Beck-Chevalley property for the fibration from
let be some other pair of convex spaces. Then the square
also has the Beck-Chevalley condition
Eigil Fjeldgren Rischel
2024
4
25
https://erischel.com/lcc-001V/
lcc-001V
/lcc-001V/
Proposition
Let be a convex function.
Then (where denotes the Legendre transform), under the identification of a finite-dimensional vector space with its double dual.
Eigil Fjeldgren Rischel2024425Proof
Recall that as a minmax problem.
Then the claim is that
Using first the rewrite and the notation
we can rewrite that as
Now observe that, restricted to the subcategory given by minmax problems where are real vector spaces, and those homomorphisms given by linear (rather than merely affine) maps, and form endofunctors, and .
Since is left adjoint to the inclusion, we have .
Hence there is a canonical map, the unit of the adjunction, for any .
If is an element of , then by the universal property, this map factors over . This gives us the inequality .
(Note that this inequality actually holds even if is not convex, and indeed we haven't really used convexity yet).
Observe that, using the natural identification , we have Clearly since the supremum is infinite unless . But observe that
Our claim now is that we may exchange these extremizers by strong duality. This amounts to the claim that the local Beck-Chevalley property holds for this square at :
But by Proposition , strong duality holds in every square of the form
and by Lemma , this is establishes that the previous square has the Beck-Chevalley condition as well, which finishes the proof.
Eigil Fjeldgren Rischel
2024
4
30
Misc stuff (sorting)
Eigil Fjeldgren Rischel
2024
4
26
https://erischel.com/lcc-001Y/
lcc-001Y
/lcc-001Y/
Composition of minmax problems
Let be a real vector space, and let be minmax problems.
Then we can try to define a composite minmax problem
For this composition to be associative relies on a strong duality property. We probably shouldn't want to treat this as well-defined unless it holds.
Note that by the convex duality stuff, the minmax problem acts as an identity for this composition.
This may fit together into a double category type structure for minmax problems (maybe restricted to linear maps between the spaces).
Eigil Fjeldgren Rischel
2024
4
23
https://erischel.com/lcc-001M/
lcc-001M
/lcc-001M/
Minmax problems are not star-autonomous
Since the category of minmax problems is very similar to a Chu construction,
we might hope that we could define a similar star-autonomous structure on minmax problems. Unfortunately, this does not work. We do have the duality, Dual minmax problem, but it doesn't extend to a star-autonomous structure.
Morally speaking, the tensor product of would be given by in the forwards direction, and pairs of affine functions satisfying in the backwards direction, with either of these expressions giving the pairing.
Since the first formula implies the pairing is convex in (as it must be), but the latter implies it's concave in , must take values only those so that is affine, and similarly for .
This can easily be an empty set, but in a star-autonomous category we always have a canonical costate , which would be impossible in that case.
Eigil Fjeldgren Rischel
2024
4
23
https://erischel.com/lcc-001K/
lcc-001K
/lcc-001K/
Proposition
The category of minmax problems has products, given by
Here denotes the coproduct of convex spaces, and is a generic element (note that )
By duality (with ), it also has coproducts given by .
Tobias Fritz
Tomas Gonda
Paolo Perrone
Eigil Fjeldgren Rischel
2023
6
15
https://erischel.com/fritz-gonda-perrone-rischel-rep/
fritz-gonda-perrone-rischel-rep
/fritz-gonda-perrone-rischel-rep/
Representable Markov categories and comparison of statistical experiments in categorical probability
Reference
10.1016/j.tcs.2023.113896
Tobias Fritz
Tomas Gonda
Paolo Perrone
Eigil Fjeldgren Rischel
2023
6
15
Abstract
Markov categories are a recent categorical approach to the mathematical foundations of probability and statistics. Here, this approach is advanced by stating and proving equivalent conditions for second-order stochastic dominance, a widely used way of comparing probability distributions by their spread. Furthermore, we lay the foundation for the theory of comparing statistical experiments within Markov categories by stating and proving the classical Blackwell-Sherman-Stein Theorem. Our version not only offers new insight into the proof, but its abstract nature also makes the result more general, automatically specializing to the standard Blackwell-Sherman-Stein Theorem in measure-theoretic probability as well as a Bayesian version that involves prior-dependent garbling. Along the way, we define and characterize representable Markov categories, within which one can talk about Markov kernels to or from spaces of distributions. We do so by exploring the relation between Markov categories and Kleisli categories of probability monads.
Matteo Capucci
Bruno Gavranović
Jules Hedges
Eigil Fjeldgren Rischel
2022
11
3
https://erischel.com/towards-cybercat/
towards-cybercat
/towards-cybercat/
Towards Foundations of Categorical Cybernetics
Reference
10.4204/EPTCS.372.17
Matteo Capucci
Bruno Gavranović
Jules Hedges
Eigil Fjeldgren Rischel
2022
11
3
Abstract
We propose a categorical framework for processes which interact bidirectionally with both an environment and a “controller”. Examples include open learners, in which the controller is an optimiser such as gradient descent, and an approach to compositional game theory closely related to open games, in which the controller is a composite of game-theoretic agents. We believe that “cybernetic” is an appropriate name for the processes that can be described in this framework.
Dylan Braithwaite
Matteo Capucci
Bruno Gavranović
Jules Hedges
Eigil Fjeldgren Rischel
2021
12
21
https://erischel.com/fibre-optics-2021/
fibre-optics-2021
/fibre-optics-2021/
Fibre optics
Reference
10.48550/arXiv.2112.11145
http://arxiv.org/abs/2112.11145
Dylan Braithwaite
Matteo Capucci
Bruno Gavranović
Jules Hedges
Eigil Fjeldgren Rischel
2021
12
21
Abstract
Lenses, optics and dependent lenses (or equivalently morphisms of containers, or equivalently natural transformations of polynomial functors) are all widely used in applied category theory as models of bidirectional processes. From the definition of lenses over a finite product category, optics weaken the required structure to actions of monoidal categories, and dependent lenses make use of the additional property of finite completeness (or, in case of polynomials, even local cartesian closure). This has caused a split in the applied category theory literature between those using optics and those using dependent lenses. The goal of this paper is to unify optics with dependent lenses, by finding a definition of fibre optics admitting both as special cases.
Eigil Fjeldgren Rischel
Sebastian Weichwald
2021
8
5
https://erischel.com/rischel_compositional_2021/
rischel_compositional_2021
/rischel_compositional_2021/
Compositional abstraction error and a category of causal models
Reference
10.48550/arXiv.2103.15758
http://arxiv.org/abs/2103.15758
Eigil Fjeldgren Rischel
Sebastian Weichwald
2021
8
5
Abstract
Interventional causal models describe several joint distributions over some variables used to describe a system, one for each intervention setting. They provide a formal recipe for how to move between the different joint distributions and make predictions about the variables upon intervening on the system. Yet, it is difficult to formalise how we may change the underlying variables used to describe the system, say moving from fine-grained to coarse-grained variables. Here, we argue that compositionality is a desideratum for such model transformations and the associated errors: When abstracting a reference model M iteratively, first obtaining M' and then further simplifying that to obtain M'', we expect the composite transformation from M to M'' to exist and its error to be bounded by the errors incurred by each individual transformation step. Category theory, the study of mathematical objects via compositional transformations between them, offers a natural language to develop our framework for model transformations and abstractions. We introduce a category of finite interventional causal models and, leveraging theory of enriched categories, prove the desired compositionality properties for our framework.
Tobias Fritz
Eigil Fjeldgren Rischel
2020
8
11
https://erischel.com/rischel-fritz-infinite-products/
rischel-fritz-infinite-products
/rischel-fritz-infinite-products/
Infinite products and zero-one laws in categorical probability
Reference
10.32408/compositionality-2-3
Tobias Fritz
Eigil Fjeldgren Rischel
2020
8
11
Abstract
We state and prove the zero-one laws of Kolmogorov and Hewitt-Savage within the setting of Markov categories, a category-theoretic approach to the foundations of probability and statistics. This gives general versions of these results which can be instantiated not only in measure-theoretic probability, where they specialize to the standard ones in the setting of standard Borel spaces, but also in other contexts. For example, applying the Kolmogorov law to the Kleisli category of the hyperspace monad on topological spaces gives criteria for when maps out of an infinite product of topological spaces into a Hausdorff space are constant.
Eigil Fjeldgren Rischel
2025
10
15
https://erischel.com/efr-AE01/
efr-AE01
/efr-AE01/
Blog
Eigil Fjeldgren Rischel
2025
10
13
https://erischel.com/efr-5NUC/
efr-5NUC
/efr-5NUC/
A simple categorical proof of (a) Martingale Convergence Theorem
Eigil Fjeldgren Rischel
2025
10
13
Theorem
Let be a sequence of probability spaces, for example for some filtration of sigma-algebras. A martingale on this object is a sequence of random variables defined on so that . Let denote the limit of this diagram in probability spaces. Then every martingale of -random variables defines a sequence of random variables on . This sequence converges in , say to , and
Eigil Fjeldgren Rischel20251013Proof
Note that defines a contravariant functor from probability spaces to Hilbert spaces and bounded maps. This functor carries the limit to a colimit. To see this, first note that it carries each map to an embedding (since every measure-preserving map is onto up to a set of measure zero), so it suffices to show that every measurable function on the limit is a limit of functions which factor over .
Let such an be given. Fix so that the -norm of the part of outside is small. Partition into intervals of width . Each of the sets is measurable. Hence there is some so that each of these intervals is approximated up to probability by some -measurable set. Define to be equal to on these approximating sets and zero outside of that. Then is within of in .
Now since forming adjoints is a self-duality on Hilbert spaces, this immediately implies that is also the inverse limit of the adjoint diagram, where the map is given by conditional expectation. It is apparent that an element of this inverse limit is precisely a martingale. Hence every martingale is the conditional expectation of a unique
This concept is essentially what is called a bilimit in domain theory and studied for embedding-projection pairs.
Eigil Fjeldgren Rischel
2023
9
28
https://erischel.com/againstcalibration/
againstcalibration
/againstcalibration/
Against calibration
Forecasting is predicting whether something will happen, like who will be the next US president or whether a natural disaster will happen or what the economy is going to be like in a year. It's notoriously difficult to think about. Typically who study this sort of thing think you should make quantifiable predictions with specific probabilities assigned to them, make them public, and then let people rate your performance later to figure out who's good at forecasting. I'm not gonna explain all this in too much detail. See eg the discussion at the start of this astralcodexten post.
One way to rate forecasts is "Brier score".[^2] If you assign something probability , and it happens, your Brier score is . If it doesn't happen, your Brier score is (since you essentially said it wouldn't happen with probability ). Lower is better - the lowest possible Brier score is , the highest is If you make many predictions, your Brier score is the average Brier score across the whole set of questions.
Another way to rate forecasters is "calibration". Calibration looks at the fraction of your "X% likely" predictions that occurred, and asks how close that fraction is to X% (for each X). Then you can plot it in graphs like this (source: Manifold on twitter):
On average, clearly, 10% of your 10% predictions should come true if you're assigning probabilities sensibly, so it seems good to have good calibration. Brier scoring agrees with this, in the following sense: suppose in fact 14% of your 10% predictions come true. Then you would have obtained a higher Brier score by replacing all your 10% predictions with 14%[^1].
[^1]: This is essentially what people mean when they say Brier scoring is a "proper scoring rule".
There are a few reason people focus on calibration
Perhaps most importantly, it's *comparable across question sets*. That is, if I make a bunch of predictions, and you make a *different set* of predictions, we can sensibly compare our calibration plots and see who did better. In contrast, predicting more difficult question sets will result in a lower Brier score even if you're doing as well as possible (in the limit, predicting coinflips, if you assign the correct 50-50 probability you'll get an average Brier score of 0.25, whereas if you predict events that happen 10% of the time and assign that probability correctly you'll get a Brier score of 0.09).
There's some evidence that calibration is *trainable* and *generalizes.* That is, if you practice, you can improve your calibration, and if your calibration is good on one type of questions (say, about politics), it'll tend to be good on other questions (about technological advances, say) as well.
These obviously make calibration a pretty useful concept, and I don't want to disparage that. It's good to practice your calibration if you want to make precise predictions, and it's good to publish your calibration graph if you want people to take your forecast seriously.
Eigil Fjeldgren Rischel
2023
9
28
Calibration is a very weak statement about the quality of your forecasts
Manifold (to their credit!) publishes their calibration plots. These are generally pretty good. Some people seem take this as strong evidence that Manifold makes good predictions.
We can try to think about perfect calibration in market terms. If some method could improve on Manifold's predictions, then that method could be turned into a profitable trading strategy. So we can think about statements that Manifold makes "good" predictions in terms of the classes of strategy they rule out. In the extreme case, if the market was perfectly efficient, no strategy could make a profit - meaning there's no way to improve the predictions.
Perfect calibration means that, (for example) of the markets currently trading at 10% YES, 10% will resolve YES. This means that no trading strategy of the type "whenever a market is valued at 10% YES, buy some of it" can make a profit in expectation. This is obviously an *incredibly weak* notion of market efficiency. We expect this to hold, but this doesn't mean the predictions are well-priced at all.
Eigil Fjeldgren Rischel
2023
9
28
Good calibration doesn't mean you have a good Brier score
Imagine two people forecasting the same set of 100 potential natural disasters. The first forecaster (call him A) assigns probability 1% to them all. The second forecaster (B) assigns probability 80% to two of them, and 2% to the rest. In the end, exactly one of the disasters happen, and it's one of the two that the second forecaster assigned 80% probability to.
A has perfect calibration. B, by contrast, has pretty bad calibration (his 80% "should" have been 50%, and his 2% should have been 0%).
But:
- B has a better Brier score (A has , B has approximately , and remember lower is better).
- More to the point, B's forecast is *clearly more useful* - having some reasonable ideas *which* of the 100 potential disasters will happen is clearly more important than getting the overall rate right
- We could make this point even more stark by letting B assign 90% probability to the disaster that ended up happening, and 1% probability to the ones that don't.
Eigil Fjeldgren Rischel
2023
9
28
Good calibration doesn't mean other people should adopt your forecasts
People often say things like "X has good calibration, meaning when they say there's a 10% chance of something happening, it happens 10% of the time. So when they say there's a 10% chance of nuclear war (or whatever), we should take it seriously".
The first sentence is of course a correct explanation of calibration. But think of our disaster forecasters from before. A had perfect calibration, meaning when A says that disasters overall happen only one time in a hundred, he's right. But if he says that some *specific* disaster has a very low, 1%, probability of happening, we shouldn't necessarily take that forecast very seriously. Perhaps it's actually easy to know which 1% of disasters happen. A's ability to get the base rate right does say something about his forecasting, but the fact that he can't do better than the base rate doesn't prove that we can't, and should therefore adopt his forecast.
[^2]: Another is log scoring, where if you say X will happen with probability and it happens, you score is instead of . For our purposes the difference doesn't really matter (both have this property of being a "proper scoring rule", so if you have to assign the same probability to some set of questions, you optimize your score by assigning the fraction of them that comes out true).
Eigil Fjeldgren Rischel
2023
9
20
https://erischel.com/coprods-lens-blogpost/
coprods-lens-blogpost
/coprods-lens-blogpost/
Coproducts in the category of lenses
Eigil Fjeldgren Rischel
2023
9
20
Introduction
The category of bimorphic lenses and its many generalizations has been
widely studied and utilized in applied category theory. We will not give
a review of the literature here, but see e.g. Riley's paper on "optics"
(one of the many generalizations) Reference , which includes a decent overview.
Dispensing with a point of notation, we will denote objects of the
category of lenses , and morphisms
as
. Note that we write
the *forwards* pass on the bottom of the tuple, breaking with the
convention used by Riley and many others.
The category of lenses
embeds into the category of polynomial functors and
natural transformations as the full subcategory of functors of the form
. Since admits all coproducts, and these
are computed pointwise, one immediate consequence is that we have the
coproducts in
. These seem to have been studied first by Hedges (Reference ). (For a reference on ,
including this inclusion, see eg Spivak Reference . is another
branch on the tree of generalizations of , studied
extensively by Spivak and others).
Given the above, one might expect these to be all the coproducts of
(up to isomorphism) - they are certainly the only ones
preserved by this inclusion[^1]. But this turns out not to be the case.
Let's begin with a worked example of the "exotic" coproducts. Note that
I simply denote by the set with
elements. In particular is the empty set.
Eigil Fjeldgren Rischel
2023
9
20
Example
In , , with the coproduct inclusions having as their forwards pass the two distinct maps , and the backwards passes in both cases being trivial.
To see this is a coproduct, consider the natural transformation
In the case where is nonempty, this is simply the map , because there are no maps , and also no maps . Clearly is a bijection. On the other hand if is empty, all the backwards passes are trivial, so we're just left with the map , which is clearly a bijection. Hence this is a natural isomorphism, so we have a coproduct.
To see why this is not preserved by the inclusion , consider the polynomial (this is the actual coproduct of the polynomials corresponding to these lenses). The inclusion maps from and into this polynomial should induce a map from , but of course there's no way to map into the (unless we map the entire thing into the component, but this clearly won't pull back to the inclusion map of when we compose with the map ). The problem is that, where in the lens/monomial case, to have any maps at all the backwards set must be , making the whole backwards pass trivial, in the polynomial case we can have the backwards set be empty only when it's forced to be.
Eigil Fjeldgren Rischel
2023
9
20
The coproducts in
Having seen the above, we may begin to lose hope of understanding the coproducts in at all. Fortunately the situation is not completely chaotic - in some sense, the preceding example is the *only* problem - all the coproducts in are either of the form (up to isomorphism), or they are of the same form as the example.
Eigil Fjeldgren Rischel
2023
9
20
Theorem
In , a tuple of lenses is a coproduct diagram if and only if the forwards passes form a coproduct diagram in , and one of these conditions hold:
For all , and for all , the function
is a bijection.
For at least one , is empty and is nonempty. (The existence of any lens then implies that must be empty as well)
The first class of coproducts are the "good" ones - these are preserved by the functor from lenses to polynomial functors, and seem to capture the correct notion of "branching" for bidirectional systems. The other class, the "exotic" coproducts, are more mysterious.
We will begin the proof of this theorem with some preliminaries about lenses in a general (Cartesian) distributive category. While we can't obtain the preceding theorem in this generality (for reasons that will become clear later), it may be useful to know what can be said here.
Let us fix a (Cartesian) distributive category - that is, has finite products and coproducts, and the universal map is always an isomorphism. I will denote the initial object by .
Eigil Fjeldgren Rischel
2023
9
20
Proposition 2.2
The functor which carries to and a lens to its foward pass admits a right adjoint, given by .
Eigil Fjeldgren Rischel
2023
9
20
Proof
We simply have to verify . But this is trivial - the forwards pass of such a lens is clearly an element of the right-hand side, so it suffices to prove that there is always a unique choice of backwards pass. But the backwards pass is a map , and by distributivity is initial. This concludes the proof.
Because left adjoints preserve (in particular) coproducts, we obtain the following:
Eigil Fjeldgren Rischel
2023
9
20
Corollary
For a tuple to be a coproduct diagram, the forwards passes must form a coproduct diagram in .
The following result, characterizing the "good" coproducts, was already essentially proven by Hedges in Reference [^2], but I'll state it here and sketch a proof for completeness.
Eigil Fjeldgren Rischel
2023
9
20
Proposition
Let be a collection of objects of , and a coproduct. Let also . Then the family of lenses with forwards passes given by the coproduct inclusions, and backwards passes by the projection , is a coproduct diagram.
Eigil Fjeldgren Rischel
2023
9
20
Proof
using the universal property of coproducts and the distributivity. This is clearly the set of tuples of lenses , and moreover precomposition with the inclusion maps clearly picks out the elements of the tuple, verifying
the universal property of the coproduct.
Eigil Fjeldgren Rischel
2023
9
20
Proposition
Let a tuple be given, such that each is the coproduct inclusion.
Given some object , each determines a map , given by . The tuple is a coproduct diagram if and only if, for all , one of these conditions hold (not necessarily the same one for each ):
1. Each is a bijection.
2. For some , the set is empty.
Eigil Fjeldgren Rischel
2023
9
20
Proof
Note that each of the lenses factors as , where the first map is , and the second map is
the coproduct inclusion.
Hence the natural transformation
factors as
where the second map is the parallel product of the precomposition morphisms for each .
The tuple is a coproduct diagram if and only if this composite is a bijection, which is true if and only if this second map is a bijection.
A product of functions in is a bijection if and only if each component map is a bijection, or one of the product sets in the target is empty (in which case the corresponding set in the domain must be empty as well for a map to exist, and so both products are empty).
We can rewrite the sets involved here as
and
This is simply the definition of
Under this expansion the map carries a lens to . So is a bijection if and only if we have empty or a bijection (or both).
Now, clearly if condition 1. holds, we've just seen that each becomes a bijection, and so our map is a bijection. If condition 2. holds, then is empty, and so we again have a (trivial) bijection.
So if at least one of the conditions hold for each , is always a bijection, and so we have a coproduct.
On the other hand, suppose there is some so that neither condition holds. Take . Then none of the mapping sets are empty, so the only way for to be a bijection is for to be a bijection, but by hypothesis this fails for at least one . By assumption none of the sets are empty, and none of the sets are empty either, so the product is not empty - hence the map is not a bijection in this case.
We can now return to the main theorem. The following argument is somewhat clunky and ad hoc. The main point is that we can characterize the cases where is empty, namely only when is empty and is not. In a general category, this can be much more complicated, and there does not seem to be any completely general method. This means we don't know which possible we need to deal with when understanding when the backwards pass is a bijection, which makes it seemingly impossible to give a general classification of the coproducts by this method.
(The second point is that it's easy to understand when the maps between hom-spaces are bijections in - we have a somewhat clunky argument but it's pretty straightforward and it can probably be done in a more conceptual way, if one took the time to find it).
We can replace with the coproduct , and each forwards pass with the inclusion, without loss of generality (if we are not in this situation up to isomorphism, we don't have a coproduct by the above)
Eigil Fjeldgren Rischel
2023
9
20
Proof of the theorem
Proof
It's clear that the first condition of the theorem implies that is a bijection always, and so we have a coproduct. As for the second condition, suppose is nonempty and is empty. Then for nonempty , the set is clearly empty, fulfilling the second condition of the lemma, whereas for empty , the map is a map between two singletons (since is empty), and hence a
bijection.
This proves that if one of the two conditions hold, we have a coproduct. To see the other direction, suppose neither condition holds. If the second condition doesn't hold, then there is always a map , and hence the set is never empty, so according to the lemma, it suffices to show that for some and some , the map is not a bijection.
Since the first condition doesn't hold, let be such that the map is not a bijection. Suppose first it's not surjective and let be something not in the image.
Take . Then the constant function is clearly not in the image of , since that would require, for some ,
for all , and in particular for , but that's a contradiction.
On the other hand suppose it's not injective, and that for some . Now take and consider a function , defined by , and for all other pairs . Now . And for all other or , clearly . Hence ( being the projection). But , so is not injective in this case. This concludes the proof.
The forwards part of this proof almost works for any distributive category with a *strict* initial object (meaning the only maps are isomorphisms, i.e has to be another initial object). The trouble is that can be initial even when neither nor is, meaning even when is initial and isn't, the nonemptyness of does not imply that is initial, meaning that we don't automatically get that all the maps are bijections. A simple example of this is something like for some natural number 1]]>. Then as long as we have empty *or* empty for each , their product is empty.
A more general class of example with the same flavor is the category of sheaves on a topological space. Then again as long as the supports of and are disjoint, their product is empty.
It may be possible to develop a generalization of the theorem, or at least of the forwards part of the theorem, using considerations like
these to control the sets of morphisms.
[^1]: There is a quibble here, since
for any sets , these being the initial objects of
. This is of course the natural "nullary" version of
the binary formula, but it also means that
, and that this coproduct
is preserved by the inclusion into , even if
.
[^2]: Hedges states it only for , but his
proof goes through unaltered for our case. He also only constructs a
particular family of coproduct diagrams, but the ones considered
here are just his composed with isomorphisms in , so
there is really nothing serious going on
Eigil Fjeldgren Rischel
2022
11
15
https://erischel.com/kelly-betting/
kelly-betting
/kelly-betting/
Thoughts on the Kelly Criterion
(Epistemic status: contains some mildly sloppy math, but is essentially true).
Suppose someone offers you the chance to bet some money on a coinflip. On a heads, you multiply your stake by , but if you lose, you lose your stake. Should you bet, and how much? Clearly this bet has positive *expected value*, and higher expected value the more your bet, but equally clearly, the actual decision depends a lot on your circumstances - and you'd be a fool to bet your entire life savings on this, or to take out as large a loan as possible to gamble on it.
One way of thinking about a bet like this is the so-called Kelly criterion. It's a particular formula for how large a fraction of your total wealth you should bet (depending on the odds of the bet), with the following nice property: if you make an infinite sequence of bets at the same odds, and bet the Kelly at each opportunity, your total profit is eventually larger than any other betting strategy, with probability 1.
Since the Kelly criterion never bets its entire bankroll, but the EV-maximizing thing is always to bet your entire bankroll, some people take this to mean there's some fundamental flaw with EV in this case - that expected value is simply misapplied to one bet in a sequence like this.
It's true that we have to be more careful here - if the thing we care about is our wealth after some sequence of bets ( going to ), we can't just assume that we should maximize EV after one bet. But if we try to maximize EV after bets, we *still* find that the maximizing thing is to bet the entire bankroll every time. But how can this be true if the Kelly dominates every other strategy with probability 1? What's going on here?
---
One way you can justify evaluating bets by their expected value is this: suppose you will get many opportunities to take a given bet, and the way the payouts aggregate is additive - that is, your total “score” that you care about maximizing is the score from each independent instance of the bet added together. Then, assuming the variance of each bet outcome is not too heavy-tailed, as the number of bets goes to infinity, the distribution of the sums approximates a normal distribution with median and variance proportional to , where is the number of bets and is the expected value.[^1]
Since for large the variance is much smaller than the expected value, the final outcome is more or less entirely determined by the expected value! This type of result is called a concentration theorem. This is a powerful justification for replacing a bet with its expected value, but only if bets combine additively!
For example, suppose the bet is something like “invest all your money in the stock market for a year”. You will get many opportunities to make this bet, but it’s not really appropriate to treat them additively - because, if you make a profit one year, you have more to invest next year, meaning you’ll be able to make an even bigger profit then, and vice versa if you lose.
Suppose a stock market investment has a chance of multiplying your investment by , and a chance of multiplying it by . This is positive expected value, but if you execute this investment many times, it’s not the case that with high probability you’ll increase your money by some amount - in fact, with high probability, your total amount of money will converge to !
We can prove this by noting that this bet aggregates by multiplication - your final score is the product of your initial bankroll, for every win, and for every loss. This means that the logarithm of your final score is the sum of the logarithm of your bankroll, log(1.5) for every win, and for every loss. Hence by the concentration theorem discussed above, the logarithm will tend toward , and since
is negative, this goes towards negative infinity, meaning your actual score must go to zero.
So IF the concentration theorem was your reason for taking EV seriously, you’re forced to work with expected log score in the case of multiplicative bets, which, if you do the math, leads you to the Kelly criterion for how much to bet each round.
BUT! Who says you should care about concentration theorems? The concentration theorem says there is at most a low probability that the log of your score is big - but if the log of your score is big, then your score must be really big! If you do the math, you’ll see that the expected value of a “bet everything every time” strategy with the odds above, given bets, is - which goes to infinity as n does. Of course, clearly most of this EV must live in some very unlikely outcomes.
When economists bring out something like the Von Neumann-Morgenstern theorem to say that rational actions are described by maximizing EV of some “utility” random variable, they’re simply not invoking a concentration theorem, so saying “the concentration theorem doesn’t apply” is just not a useful knockdown, and it doesn’t mean that utility maximization theory fails to account for this type of repeated bets. Of course, utility theory doesn’t rule out the Kelly betting behavior either - Kelly betting is perfectly rational, explained by utility that’s logarithmic in money (or whatever the bet is being made in).
[^1]: The mean, that is to say the sum of the first samples divided by , always converges to the expected value with probability , this is "the law of large numbers". Of course maximizing and maximizing amounts to the same thing, but you probably should care about the fact that the variance is growing with !
Eigil Fjeldgren Rischel
2022
6
4
https://erischel.com/approximate-categories/
approximate-categories
/approximate-categories/
An approach to approximate category theory
A number of different people have thought about ways to bring notions of
approximation into category theory. There seem to be essentially two
notions that one would like to express here:
1. The idea that a digram, while it may not quite commute, commutes *up
to some specified tolerance *
2. The idea that a mapping, while it may not quite preserve the
relevant structures, preserves them *up to some specified tolerance
*.
The first idea is, at a basic level, captured by categories enriched in
(some monoidal category of) metric spaces, whereas the other can be
captured by categories enriched in some category of sets with a function
, again equipped with a monoidal product suitable for
the purpose at hand.
Categories enriched in metric spaces can be weakened to the *metagories*
of Tholen and Wang (link) - these are, essentially, graphs with an "area"
measure , defined whenever ,
and form the edges of a 2-simplex, satisfying
some sort of resonable 2-dimensional analogue of the triangle
inequality. The basic idea is that the area measure indicates the
failure of the given triangle to commute. So if , is a
suitable choice for . The axioms correspond to unitality and
associativity. They also imply that
1. The set of maps has a pseudometric[^1]
2. Composites in the sense above are well-defined up to distance zero,
i.e given two choices of composite they have .
Of course, given a category enriched in metric spaces, you can take
.
The second idea seems to be Paolo Perrone is discussing in this talk, calling categories like this "weighted categories".
I want to spell out here a common generalization of the two approaches
to "quantitative category theory" that I haven't seen discussed anywhere
else. I think, beyond the convenience of having a formalism capturing
both ideas, it is in fact natural to do so - certain operations, natural
to a category theorist, take you from one of these worlds to the other,
in a way that's neatly described by this approach. That being said,
there are still some issues, which I'll also discuss.
The basic approach is to consider a type of *filtered simplicial set*,
i.e a functor .
Let's write this as . We will probably want to
assume that the maps for
are all injections. Such an object is almost the same thing as a
simplicial set (defined by
) where
each simplex has an associated real number
, given by the first where that simplex
appears in , such that the
face an degeneracy maps never increase . This is not quite correct,
because for each simplex the set of such that
can be either or for
some . We will probably want to impose the further condition that
it's always the former. This amounts to the claim that
is the intersection of for all r]]>, which is a sort
of sheaf condition.
Okay, so the basic setup now is that we have a simplicial set, where
each simplex has some associated positive number. It will be useful to
think of this as the *error* of the simplex. Let me give a few examples
of this to illustrate how I want to use this structure:
1. Given a category enriched in metric sets , we can
consider the filtered simplicial set where
- for each .
- is the set of morphisms in , again
regardless of .
- consists of triples
so that
.
- Each is -coskeletal, i.e given a compatible
boundary for a higher simplex, there is always a unique such
simplex (at any given error level).
It's clear that this data characterized such a category up to
equivalence, although it's not totally obvious how to characterize
the subclass of filtered simplicial sets which have this form.
2. Given a category equipped with a *weight* in the sense of Perrone,
i.e a number for each morphism such that ,
, we can build a filtered simplicial set as
follows:
- is the set of objects, for all .
- is the set of morphisms with .
- is the set of commuting triangles with
.
- Each filtration degree is coskeletal, as above.
Again, it's clear that we can pull out a category with a metric on
it in a unique way from a filtered simplicial set like this.
Thus, you're supposed to interpret a 2-simplex in as
saying "this triangle commutes, at least at error tolerance ", a
morphism in as being "a morphism up to error/metric
". In principle you could also have objects with errors, and higher
simplices with errors if you wanted to do higher-categorical stuff.
Let's think about what makes such a simplicial set suitable for use as a
category. Recall that an ordinary simplicial set is the nerve of a
category if and only if every inner horn has a
unique filler, which is in turn equivalent to asking that horns of every
dimension have unique fillers, or that the maps
X[n] \to X[1] \times _{X[0]} X[1] \times _{X[0]} \dots \times _{X[0]} X[1]
are all bijections.
In other words, this is about fillers for certain horns existing, and
perhaps existing uniquely. The world of quantitative categories is a bit
more complicated, for the essential reason that
Here is my attempt at a suitable definition. For each
, and ,
let be the filtered simplicial set
given by a horn , with the th face for each
having value , i.e appearing in
but not before (with
the lower-dimensional faces all having the maximal possible value given
this).
Similarly define (given a full list of
values). Then we say a filtered simplicial set is a
*approximation quasicategory* if, for every , and each tuple
every map
admits an
extension along
.
This obviously corresponds to the definition of quasicategory (aka
-category, weak Kan complex...), with the additional structure
of the filtration incorporated in such a way that errors add when
composing morphisms. This is perhaps a good time to note that one could
also have combined errors with instead of .
This would have amounted to asking that each
be a quasicategory in itself. From a
categorical point of view this condition is much more natural, but
adding errors seem more natural from the point of view of metrics,
incorporating the triangle identity.
It is not entirely clear to me whether requiring each filler to exist
uniquely, rather than merely exist, captures the right notion of
1-category. The reason is that it demands that composites be unique,
whereas we might expect (from our earlier look at metagories) that they
should only be unique up to an induced metric-0 notion. I tend to think
that it's more sensible to require this equivalence relation to be
already quotiented out, to make that part of the definition of
approximation quasicategory, but I'm not entirely decided on this point.
1. The category of morphisms in a
metric-enriched category naturally acquires a nontrivial filtration
in the morphisms (which are commutative diagrams).
2. Iterating this, the category of morphisms in such a category
naturally acquires a nontrivial filtration in the *objects*.
Working out all of category theory in this context is a big undertaking,
which will probably not get done until people derive more actual utility
from it. However, as a sort of test case to explore this system, I've spent some time thinking about how colimits should be defined, which I hope to explore in a future post.
[^1]: A metric without the requirement that implies
Eigil Fjeldgren Rischel
2022
3
28
https://erischel.com/20220325115704-frag-coarse-spaces-have-a-nice-group-theory/
20220325115704-frag-coarse-spaces-have-a-nice-group-theory
/20220325115704-frag-coarse-spaces-have-a-nice-group-theory/
Fragmentary-coarse groups are categorically neat
Eigil Fjeldgren Rischel
2022
3
28
Coarse geometry
The idea of a _coarse space_ is to formalize a sense in which the inclusion of metric spaces is an equivalence.
These two spaces have the same "large-scale structure", in the sense that any function into can be approximated up to uniformly bounded error by one into . Say two functions into a metric space are _close_ if there exists a constant with for all . Say a function between metric spaces is _controlled_ if postcomposition with it preserves closeness (it turns out there is a simpler equivalent statement of that, but it doesn't really matter). Then we can consider equivalence classes of controlled functions between metric spaces, under the equivalence relation of closeness. This is "coarse geometry".
It turns out there is a way to abstract away from the metric and describe a "coarse structure" on a set, which remembers enough information to tell which pairs of functions are close and which are controlled, but nothing else - and that there then exist coarse structures that aren't induced by metrics. I personally think of this as kind of complementary to the way topologies remember what's necessary to talk about continuity - if continuous functions between metric spaces are about preserving the "small-scale structure", the notion of convergence, then coarse geometry is the opposite of that.
The category of coarse spaces admits finite products, so we can talk about "coarse group theory", which is explored in An Invitation to Coarse Groups by Leitner and Vigolo, which I've been reading a bit recently. It's full of interesting stuff - maybe my favorite bit is the story about geometric group theory. If you have a group with a generating set , you can define a metric by setting to be the length of the shortest way of writing as a word in generators - this amounts to taking the graph distance on the Cayley graph. Studying this metric structure on the group can tell you a lot of interesting stuff about the group - but the metric is highly dependent on the choice of generating set ! So it's somehow very surprising that all these different metric come back and give us the same group-theoretic information (since the group is fixed). Leitner and Vigolo shows that for all finite , the metrics give the same coarse structure on , and much of geometric group theory "sees" only coarse structure!
I find all this really cool and I highly recommend the paper. I want to talk about a subtlety that comes up in the category of coarse groups.
Eigil Fjeldgren Rischel
2022
3
28
The problem: group homomorphisms lack kernels
Consider two possible coarse structures on the set - the trivial structure and the one induced by the normal metric, which we can write . The identity map is obviously a coarse group homomorphism. It's also an epimorphism. In normal group theory we have the very convenient isomorphism theorem saying that the image of a group homomorphism is, up to isomorphism, the quotient of the domain by the kernel. But there is no coarse subgroup of that can play this role.
Let's think about how to remedy this. The universal property of the kernel dictates that maps into it must be precisely those maps into that are close to zero in . The first condition boils down to taking each coarsely connected component to the same point, and the second condition means that the image must be a bounded subset (in the metric). Of course, there is no coarse set like this - it would have to consist of "the bounded subsets of ", but of course every point is contained in one of these subsets.
Eigil Fjeldgren Rischel
2022
3
28
The solution: relaxing the notion of coarse space
The answer is to relax the notion of coarse space, by removing the condition that is among the controlled sets.
This leads to the notion of _fragmentary coarse space_:
- A fragmentary coarse space is a set equipped with a family of subsets of , stable under finite unions, subsets, and so that if , then .
- A subset is called a _fragment_ if is in .
- Two functions between fragmentary coarse spaces are _fragmentary close_ if, for each fragment of , .
- A function is _fragmentary controlled_ if, for each , .
- The category of fragmentary coarse spaces and equivalence classes of fragmentary controlled functions up to fragmentary closeness is denoted
Leitner and Vigolo show that is complete and cocomplete, and in fact even Cartesian closed (these convenient properties are the reason they introduce the notion of frag-coarse space). I now claim that it is furthermore _regular_.
Observe that this property passes to the category of group objects in . Hence given any quotient homomorphism between (fragmentary) coarse groups, there exists a kernel so that , although the kernel may have to be a fragmentary-coarse group even if and are coarse. Thus, passing to fragmentary-coarse groups provides us with a nice group theory.
Eigil Fjeldgren Rischel
2022
3
28
Proof that is regular
Since is complete and cocomplete, it suffices to show that pullbacks of regular epimorphisms are again regular.
We first claim that every epimorphism is in fact regular.
Note that Leitner-Vigolo show that is an epimorphism if and only if, for each fragment of , there exists a fragment of and a controlled set , so that (i.e for every , there exists so that ).
Given this, we see that is frag-coarse-equivalent to equipped with the frag-coarse structure - clearly is a frag-coarse surjection from this space to , and also a frag-coarse embedding, hence an equivalence.
So every epimorphism is, up to isomorphism, given by a map which is set-theoretically the identity. One also sees that the codomain necessarily has the same fragments as the domain.
Now, inspecting the construction of coequalizers for frag-coarse spaces, it's not too hard to see that the frag-coarse structure on that gives the coequalizer of the kernel pair of must coincide with the one on the codomain on . Hence every epimorphism is regular.
Now, if is an epi (assumed to be set-theoretically the identity), and is any map, we must show that again is an epimorphism. The construction of limits makes it clear that the fragments of the pullback are each contained in a set of the form where are fragments of and respectively and is controlled in .
Since for each fragment of , is a fragment of , it's clear that we can find a matching fragment for each . Then the projection from this fragment is surjective on the nose, and hence the projection is epimorphic.
Now let's consider the example of . In fact since the forgetful functor from group objects preserves all limits, we can compute the kernel of this map just in . It is given by the frag-coarse structure on where a set is controlled if it's bounded (namely if the identity is close to zero on it) - not if the distance is bounded as ranges over , but if the whole thing is a bounded subset of in the -norm. This means that the fragments are exactly the bounded subsets of (and that there is no nontrivial frag-coarse structure beyond that - a set is controlled if and only if it's contained in the product for a fragment ).
It's not too hard in this case to verify that the quotient of by this subgroup is in fact . Namely, a coarse map out of restricts to zero on the subgroup if and only if it carries the bounded subsets of to sets that are bounded-close to zero in the codomain, i.e sets so that is controlled. But for a coarse group homomorphism, this is enough to be controlled as a morphism on .
Eigil Fjeldgren Rischel
2022
1
26
https://erischel.com/links-2022-01-26/
links-2022-01-26
/links-2022-01-26/
Links 2022-01-26
Astral Codex Ten: Bounded Distrust, also Against That Poverty And Infants EEGs Study
In Search of Visual Texture
How should you talk to think better?
Tweet
Eigil Fjeldgren Rischel
2022
1
22
https://erischel.com/links-2022-01-22/
links-2022-01-22
/links-2022-01-22/
Links 2022-01-22
Dordle, a Wordle variant where you have to guess two words at the same time.
Intercats, a new seminar from the Topos Institute on "categorical interaction". I'm scheduled to speak here (in June, so don't get too excited yet)
How To Become A Magician. See also Becoming A Magician.
Postmortem on RatVac. I have the highest level of respect for everyone who made their own vaccine - major props.
Compositional Thermostatics from Baez, Lynch, and Moeller. I can also recommend Owen's blog.
Eigil Fjeldgren Rischel
2021
11
13
https://erischel.com/smooth-dynamical-systems-as-infinitesimal/
smooth-dynamical-systems-as-infinitesimal
/smooth-dynamical-systems-as-infinitesimal/
Smooth dynamical systems as infinitesimal discrete dynamical systems
Here I am working with nonstandard analysis in the sense of Robinson, taking an ultrapower of the real numbers and building things out of that. But in general I am going to be a bit sloppy and not worry too much about the details.
Recall that a standard function is _differentiable_ if, for every standard and for every infinitesimal , , where is also standard.
Fix an infinitesimal .
Let be a standard differentiable manifold, fix , and consider the tangent space . Recall that the elements of this vector space (may be taken to be) smooth paths , with some open interval, up to the equivalence relation of having the same first derivative at (which may be checked in any choice of local coordinates, all coordinates giving the same answer).
In particular we may compute this derivative, letting , being the local coordinates of choice, as the standard part of .
Having fixed , we may thus replace each curve with just the choice of , the only condition on this being that in any choice of local coordinates, the above quotient has a standard part (eg it is not unbounded). But since the change-of-coordinate maps are in particular differentiable, it suffices to verify this for one such choice.
We may call this property "being at -order infinitesimal distance from from ".
For example, suppose that , and . Then if , it will not do to take , for in that case we will get the unbounded nonstandard real for the difference quotient.
Let be the set of such points in . We don't quite have a map , because e.g the path , which has local derivative zero, has . In order to make this map well-defined, we need to consider a quotient by the relation of "being at distance ". More precisely, let if, if any coordiante chart, is infinitesimal. (The local differentiability ensures this does not depend on the choice of coordinates.) Then let - now the map is well-defined.
We can also define a vector space structure on that makes this map linear.
We simply lift the addition and scalar multiplication from any coordinate chart.
Local differentiability imply that , wher is a change-of-coordinate map, so that addition is well-defined up to an infinitesimal times , which is quotiented out by in any case. Scalar multiplication works similarly.
Note that we are defining a vector space over , _not_ over the full . Multiplication by a number of order is not invertible (in fact such products are always zero), and similarly scaling by an unbounded number may take you out of the coordinate patch, and is thus not well-defined.
In fact with this, we have an isomorphism - any is the image of , and two curves have the same derivative exactly if .
Recall that smooth dynamical system on is a smoth section .
We can exploit our isomorphism above, by observing that each is actually a quotient of a subset of . Hence we may ask for a function so that - this induces a smooth dynamical system. Every (standard) smooth dynamical system has this form, and induce the same system if for each .
What's cool about this is that these are essentially _discrete_ dynamical systems, albeit nonstandard ones. A discrete dynamical system is a set with an "advance one timestep" function . So a smooth dynamical system is a manifold with an "advance time " function , subject to the condition that is -close to , and up to a certain equivalence relation. This suggests a way to use the same conceptual tools to study smooth and discrete dynamical systems.
The paper Differential geomtry via infinitesimal displacements, by Nowik and Katz, provides a more in-depth analysis of this idea.
Eigil Fjeldgren Rischel
2021
6
19
https://erischel.com/20210618162423-martin_lof_random_sequences/
20210618162423-martin_lof_random_sequences
/20210618162423-martin_lof_random_sequences/
Martin-Löf Random Sequences
A sequence of bits is _Martin-Löf random_ (or _algorithmically random_) if, roughly speaking, there is no _computable_ pattern to it.
There are three equivalent definitions:
Eigil Fjeldgren Rischel
2021
6
19
Kolmogorov complexity definition
Let be the kolmogorov complexity of a binary string (finite).
Say is -incompressible if . An infinite string is Martin-löf random if there exists so that all its finite prefixes are -incompressible.
Eigil Fjeldgren Rischel
2021
6
19
Constructive "covers"
Given a finite string , is the open set of strings with that prefix.
A constructive open is an open given as the union of an enumerable sequence of such things.
A constructive nullcover is a computable sequence of constructive opens so that ( being the obvious "lebesgue" measure on Cantor space).
A sequence is Martin-Löf random if, for any constructive nullcover, .
Eigil Fjeldgren Rischel
2021
6
19
Constructive Martingales
A _Martingale_ is a function , interpreted as the profit of a betting strategy (betting on coinflips) on the given sequence of coinflip outcomes.
It's _fair_ if - in other words, the amount you win on a zero must equal the amount you lose on a one. Ie "you're betting at fair odds".
A martingale _succeeds_ on a Martin-löf random sequence if its limsup on the prefixes is infinite - i.e if it wins unbounded amounts of money at fair odds without ever risking any more than its finite starting pot (since it cannot go negative).
A martingale is _constructive_ if it is "lower computable", and a sequence is Martin-Löf random if there is no constructive martingale that succeeds on it.
Ie if there is no computable gambling strategy to extract unbounded money from the assumption "this sequence produces fair coinflips".
Eigil Fjeldgren Rischel
2021
6
19
Discussion
In other words, as long as the world is computable, you can comfortably treat any Martin-löf sequence as random. Of course, if the world is computable, you cannot store one in any way, and any black box that outputs a Martin-Löf sequence is in some sense "either random or uncomputable".
So in some sense there's _no way_ of telling the difference between a deterministic, computable world that contains some black boxes that output Martin-Löf sequences, and a computable world that contains some black boxes that output "actually random" sequences.
This provides some justification for the philosophical position that "randomness" is always about quantifying the limitations (of information, or in this case, of which functions they can compute) of the observer, advanced eg here.
Eigil Fjeldgren Rischel
2021
6
19
Proof (sketches) of equivalence
Suppose belongs to some constructive nullcover,
Then we can construct a constructive martingale which essentially makes the bet " will be in ", continuously making a profit.
Basically, assume we have already defined for all the strings up to length ,
Consider a string of that length. Either both of and belong to , or neither, or exactly one of them.
If neither do, then we know we're not looking at , so it doesn't matter.
If both do, then we make no bet at this point, but use again to compure and so on.
If only one of them do, then we bet all our money on that one.
This martingale succeeds on , and is constructive because the s are.
On the other hand, suppose a constructive martingale succeeds on .
Without loss of generality, assume , where is the empty string.
Let be the set of strings where 2^n]]> for some prefix.
Clearly no fair betting strategy can get positive expected value, so the measure of is at most . On the other hand is computable because is. This proves equivalence of the martingale and constructive nullcover definitions.
Now suppose is compressible, i.e the value is unbounded.
Then there exists a program which receives and infinite bitstring as input and produces output one bit at a time, and an input , so that when run with input it produces as output, and so that the difference between the number of input bits consumed and the number of output bits produced grows without bound (the input is just a suitable sequence of very effective compressions of . The program simulates these, then outputs the extra bits from the end).
Note that given , we can compute so that, after consuming bits of input, the program produces bits of output.
Then we can define to be the set of infinite strings that have a length prefix which is a possible output of the program on an input of length .
There are at most such outputs, each of which determines a set of measure , so this has measure . But clearly is in there.
On the other hand, suppose is in some nullcover .
Then we can try to compress the prefixes of as follows:
- Compute the gödel number of the program enumerating the prefixes defining , for some
- Find the th output of this program, for some , which is necessarily a prefix of . We may assume wlog that this program produces its output in ascending order of length.
The length of this program is a constant , to store the program computing and for gluing code, plus bits to store and .
Let be the length of the th prefix defining
Then .
This implies that the is at least .
Hence this is a length program that produces a prefix of of length at least - hence for large enough, this is an arbitrarily good compression.
Eigil Fjeldgren Rischel
2021
4
25
https://erischel.com/example-of-reading-process-cellular-sheaves/
example-of-reading-process-cellular-sheaves
/example-of-reading-process-cellular-sheaves/
Example of my reading process: Cellular sheaves of lattices and the Tarski laplacian
There's a lot of sort of "implicit" skills that are important in various fields of science, that you really only learn by just hanging around older people that already know them and picking things up by osmosis. This is one of the things that make it hard to just learn things by reading textbooks, as opposed to actually going to a university and getting a degree. I think we should be doing more to study and teach these sorts of skills, to the extent that it may be possible.
Therefore, I've decided to write this post where I describe my "process" for reading papers by means of example. The paper I'll be reading is a random one that came across my feed[^fn:1]: Ghrist and Riess: Cellular Sheaves of Lattices and the Tarski Laplacian.
I got the idea for this project from Alexey Guzey tweeting this:
This post is an "async" version of Alexey's request - although as he notes in the replies to that tweet, the difference in learning rate between "reading what someone wrote about doing something" and "interacting with them as they do it live" is pretty insane. So this post is also an offer: if anyone wants to hop on a video call with me and hang out while I read a paper, let me know! My email is ayegill (at) gmail (dot) com.
Some notes before we begin:
I selected this papre by skimming the abstract and deciding it looked interesting. Once I'd decided I would use this paper for this post, I committed to "finishing" reading it even if it turned out to be not that interesting - normally I might have skimmed it and decided not to keep reading it.
"Finishing" is obviously still pretty variable - some papers I will read in a lot more detail than this one.
My process involves a lot of being distracted by twitter, having to go do something else, and so on, which has been elided in this description.
Eigil Fjeldgren Rischel
2021
4
25
My process
The very first thing I do is to send the paper to my [reMarkable]. I read the paper on the reMarkable, and keep a piece of paper next to me for scratches, and my laptop open for googling and for notes.
I go over the beginning of the paper, taking some loose notes.
I jot the following down on a piece of paper:
**Terms**
- Reeb graphs
- The thesis of curry
- Cellular sheaves? How do they work?
- (Unwritten thought: I remember that I've read about "cellular sheaves" before. They're some sort of system for encoding a sheaf on a cellular complex of some soty, in a way that's "analogous" to sheaves on the geometric realization on the complex.)
- Hodge Laplacian for vector spaces?
- (Unwritten thought: the paper is about a laplacian for lattices - this seems to be a different version of the same concept? What's that like?)
- Graph signal processing?
- (Unwritten thought: the paper described this as signal processing where the signals live on graphs instead of in real numbers?)
I also make notes when something makes me think of an idea related to one of my projects. At this point I'm at page 7, and I've made two such notes thus far.
At this point I have a vague idea of where the paper is situated - what's going on. There's a lot of references in this introduction to ideas I don't really know about, or only half remember. But nothing has made me think I should go look it up before I proceed.
At page 7 or so is where I get impatient with the paper just dumping a bunch of definitions around lattices on me, and start skimming forward a bit. On page 8, I notice the important point that we're dealing with categories of lattices where the morphisms are _connections_. Here I remember that I already know about galois connections - a connection being the same thing but here they don't reverse the order.
On page 9 we get to "cellular sheaf theory". Here the authors luckily recall the definition of cellular sheaf I was missing before. I make a note in the margin at the top of page 10: "Ie - **not** the other way".
This is the important point of a cellular sheaf
- It assigns an object (vector space,set,etc) to each _cell_ in a cell complex
- The restriction maps go the opposite way from what you expect, from points to segments (and in general to higher-dimensional cells), not the other way around
- There is no sheaf condition.
I briefly try to connect this picture with something I half-remember about the connection between a space and the geometric realization of the nerve of a cover (while reading the paper, I didn't remember the term Cech nerve, but just had a picture in my head which upon reflection seems to be captured by that term). I fail to make any actual progress with this but mentally note that this idea smells right. I skim the description of global sections and so on, but note the definition of the cellular cochain complex.
In the section on Cellular Hodge theory, I make the following note in the margin: "every de Rham class has a unique harmonic rep", summarizing what they're saying about the kernel of the Laplacian for Riemannian manifolds (/what I half-remember about this stuff). "Harmonic" here means exactly "in the kernel of the Laplacian". I make a note of the "hodge laplacian".
At this point I am skimming forwards a lot. I stare at the Tarski Laplacian on page 12 for a bit. I write "how does this 'diffusion' work?" in the margin. Then I think a bit about how to view this stuff as "diffusion".
I think something like the following:
- The expanding part is the completion associated to the connection
whenever is an edge adjacent to , then the "intersection" of all those completions for every edge.
The the mixing part is kinda this same thing, but for every _neighbor_ vertex, i.e for every edge from a different vertex , you "extend" from to the edge, then restrict to . "Mixing".
I read some of the stuff about how diffusion lets a cochain "flow" to a global section, and made a note of the similar property of the Tarski laplacian - that the fixpoints of are exactly the global sections. Here I also thought to myself that it was interesting that this is just a _zero-dimensional_ theory, no higher homology objects.
Now I skim forward a bit more to page 15 at the bottom, "Tarski Cohomology", where the authors get to the question I just raised (this sort of "author mindreading" is always pleasing). They note here essentially that the definition for dimension zero also works in higher dimensions.
Then they pass to the comparison with more "ordinary" chain complex based cohomology. Here I initially skim forward and try to get the gist of things. I see they introduce something called the "Grandis cohomology", which I note is cohomology of a chain complex in lattices (which is in fact a preadditive category, so this makes sense kinda). You can compare this with usual cohomology by considering the "Grassmannian functor" associating to a vector space its lattice of subspaces. Here I got a bit confused about the relationship between the various types of cohomology, so I had to go back and forth a bit to make sense
- The Tarski cohomology is the fixpoints of a certain endomorphism on the -chains (defined in a natural way)
- Given a chain complex, the Grandis cohomology is the cohomology in the usual sense
- We can compare these in the case of a sheaf of vector spaces, by either
- Constructing the usual chain complex of vector spaces, usign the Grasmannian functor and taking grandis cohomology
- Or taking the associated Grassmannian sheaf and then taking its Tarski cohomology.
These are not in general the same.
The reason you can't just build a chain complex out of a sheaf of lattices is that, while you can define a coboundary map, it doesn't form a chain complex, in the sense that .
We can however do still do something with this fake chain complex, apparently involving mimicking the definition of the Hodge laplacian. This gives another type of cohomology, which is also not equivalent to Taski cohomology.
At this point there are still a lot of details I don't understand. I have the big picture, though. The next step would probably be to pick up some of the references to understand why this is useful - but I decide not to do that.
I also consider making flashcards or notes about this paper. This blog post is already a file in my notes, so I have that. I decide that these concepts _probably_ aren't worth comitting to memory, except "cellular sheaves", which I've come across a few times. I also decide to make notes on some of the concepts.
If you want to look at the sparse margin notes I took while reading this paper, it's here.
[^fn:1]: In this case, it was recommended to me by <https://arxivist.com>
Eigil Fjeldgren Rischel
2021
4
25
https://erischel.com/this-weeks-finds-in-act-20210425/
this-weeks-finds-in-act-20210425
/this-weeks-finds-in-act-20210425/
This Week's Finds in ACT - April 25th
I wrote a long post on one of the papers I read this week: Example of my reading process: Cellular sheaves of lattices and the Tarski laplacian.
See that post for the details!
Some other stuff:
Eigil Fjeldgren Rischel
2021
4
25
Jade Master: The Open Algebraic Path Problem
This paper came out in 2020, so it's practically ancient history.
But it's really cool! It's about
- The problem of finding a path between two vertices on a graph
- An algebraic generalization of this where you replace "graph" by more exotic things (the "algebraic path problem")
- An _open_ generalization of this where you build your (generalized) graph by gluing together smaller examples.
The basic algebraic structure is a _quantale_ - an ordered set equipped with a binary operation (compatible with the order), which has all joins.
Some basic examples:
- with the operation (OR). Here the algebraic path problem detects the existence of a path from one vertex to another.
- equipped with and given the _reverse_ order (so join = infimum). In this case the algebraic path problem finds the length of the _shortest_ path ( if there is no path).
The first observation is that this boils down to computing the pointwise join of all the powers of the adjacency matrix corresponding to a graph - where we simply _define_ a "graph over a quantale" to be such an adjacency matrix. If the quantale is , this is a graph in the usual sense, each edge is either there or not. If the quantale is , each edge has a certain length (which may be infinite to denote "no edge"). Thus we have "the algebraic path problem".
The second observation is that you can "glue graphs together" by taking pushouts in a certain category of matrices, and this respects the computation of the paths above. There is a lot of very nice category theory in this.
Eigil Fjeldgren Rischel
2021
4
25
Tom Leinster: The Magnitude Of Metric Spaces
This is an even older paper, which spawned a lively field of research (see eg here and here). This is about associating an invariant called _magnitude_ to metric spaces, which measures their "size" in a kind of hard-to-understand way:
- The magnitude of a finite set of points, all at distance , is the number of points
- As the distances go to zero, the magnitude converges to one (in the above case of finitely many points)
- The magnitude of the interval is .
I don't think I really understand this stuff deeply, but it's very interesting!
Eigil Fjeldgren Rischel
2021
4
18
https://erischel.com/this-weeks-finds-in-act-20210418/
this-weeks-finds-in-act-20210418
/this-weeks-finds-in-act-20210418/
This Week's Finds in ACT
John Baez wrote a regular blog/column called "This Week's Finds in Mathematical Physics" circa 1993-2012. The entries are really a treasure trove of cool mathematical nuggets, covering everything from hardcore theoretical physics, group theory, climate models, category theory, and more.
Imitation being the sincerest form of flattery, I decided to shamelessly steal this format, and so this is hopefully the first of many "This Week's Finds in Applied Category Theory". Below, I've summarized a few of the papers/blog post/notes/whatever I read this week. I didn't stick religiously to things I first came across this week, and indeed some of these are pretty old, but they're all things I spent some time mulling over this week.
Eigil Fjeldgren Rischel
2021
4
18
Tom Leinster: Algebraic Closure
Galois theory is one of the cornerstones of algebra.
It studies the relationship between _field extensions_ - that is, inclusions of one field in another - and subgroups of the group , of those automorphisms of with for all .
It's somewhat clunky to treat this topic using category theory, for the simple reason that a lot of constructions aren't functorial, involving some choices that can't be made canonically.
Tom Leinster explains a neat proof of one of the fundamental theorems, namely that all fields admit an algebraic closure. The trick here is to work with _rings_ for as long as possible, and then right at the end quotient by a maximal ideal to pass back to fields.
The post ends with a remarkable idea.
The algebraic closure is not functorial, in the sense that there is no endofunctor and natural transformation so that is always the inclusion of in its algebraic closure.
However, Leinster conjectures that this can be remedied as follows:
Consider a pair , where is a topos and is a field in - meaning a ring satisfying the formula . Then we can consider the collection of morphisms to pairs , where is algebraically closed. Then for , a normal field, there is an initial such map, given by the topos of sets with an action of the absolute Galois group of , and equipped with its natural action.
Unfortunately, this doesn't hold.
To see why, first recall why the map of ordinary fields is not initial among maps from to algebraically closed fields. The reason is simply that is not trivial, so there are multiple maps from to .
Now let be an element of the absolute Galois group, and consider the functor that carries a set to itself with the -conjugate -action, with (here is the original action).
This is an equivalence of categories, so certainly a geometric morphism (even "in both directions").
Now is a map "over" the inclusion . It's obviously not -equivariant - but it _is_ equivariant from .
Namely, , which is exactly what should hold.
Thus, both the identity and form maps
over
(To verify commutativity of the geomtric morphisms, the functor equips a set with the trivial action, and obviously conjugating the trivial action you still get the trivial action). Just as in the normal case, this contradicts initiality.
Eigil Fjeldgren Rischel
2021
4
18
Dan Shiebler Categorical Stochastic Processes and Likelyhood
In categorical approaches to probability, we usually study categories of stochastic maps given as Kleisli categories of "probability monads" - the ur-example being the Giry monad which carries a measurable space to the space of probability measures on it.
This is quite distinct from how something like a stochastic process is usually treated in probability theory. There one would usually consider a function , where is some "background space of samples", equipped with a probability measure.
This formulation is strictly more expressive, since it allows you to express correlations between and , but since this is exactly expressive power that you usually _don't_ want in the usual categorical setups, in some sense the Giry formulation may be more appropriate.
In any case, Dan Shiebler develops this alternative approach in this paper.
He also considers how the paradigm of _maximum likelyhood_ statistics fits into the categorical learning framework as developed eg by Fong-Spivak-Tuyeras in Backprop as Functor and further by Crutwell-Gavranovic-Ghani-Wilson-Zanasi, Categorical Foundations of Gradient-Based Learning
Eigil Fjeldgren Rischel
2021
4
18
Composing Open Dynamical Systems 2: Undirected Composition
A blog post from the AlgebraicJulia project, about implementing certain ideas from ACT in actual software.
Here, they're discussing a notion of "composing dynamical systems".
There are different ways of doing this:
- If you have a _parameterized_ dynamical system - one which depends on certain inputs - you can let another dynamical system control those inputs
- If you have two dynamical systems, both with a distinguished variable of the same type, you might be able to have them _share_ that variable. This requires that you can somehow "add up" the changes prescribed by each of the dynamical systems. For example, for ODEs you can add the derivatives, and for "difference equations" like , you can add the differences. But of course given a fully general "discrete dynamical system" like , you can't do this.
The blog posts discusses the implementation of this idea in the `AlgebraicDynamics.jl` package.
Eigil Fjeldgren Rischel
2021
4
18
Realizability as the Connection between Computable and Constructive Mathematics
You probably know that _intuitionistic_ logic is where you don't assume the Law of Excluded middle, for all formulas .
You may have heard this described also as _constructive_ logic, and heard something along the lines of "to prove an existence statement constructively, you have to provide an explicit algorithm for constructing an example".
Since there are many interesting models of intuitionstic logic that refute LEM, which have nothing to do with algorithms (like toposes), this statement is obviously a bit odd. The right way to interpret it is _realizability logic_, which formalizes the idea of "there is a computable procedure verifying this statement". Then we find that
- In order for to hold, there must be an algorithm that produces such that (in a suitable sense)
- The logic obeys the rules of intuitionistic logic (but not in general LEM, because this would require that for any statement, there was an algorithm which figured out whether or holds, which is not true).
Eigil Fjeldgren Rischel
2021
3
28
https://erischel.com/march-2021-links/
march-2021-links
/march-2021-links/
March 2021 Links
Also contains links from February.
Fantastic Anachronism: Two Paths to the Future
Ansuz: What color are your bits? What do the notions of "random number" and "copyrighted music" have in common? They're not about the specific bits under consideration, but about their _color_.
mike_hawke: Some random parenting ideas.
Zvi: Why I Am Not In Charge
I was very taken with Lucy Greer's blog drossbucket in general, and I particularly enjoyed this recent post: Speedrun: "Sensemaking", where she tries find out as much as she can about this nebulous term in one hour. Seems like a cool thing to emulate!
Michael Nielsen: Maps of Matter.
Derek Sivers: There is no speed limit
Autotranslucence: Becoming a Magician
Higman's embedding theorem states that a finitely generated group can be embedded as a subgroup of a finitely presented group precisely if there is a presentation where the relations are recursively enumerable, i.e there is a computer program that generates them one after the other.
Of course this statement makes sense if you replace "group" here by any single-sorted algebraic theory (i.e Lawvere theory) - The Boone Conjecture is the conjecture that it's always true.
This is now known to be false in general, but of course the problem of characterizing those algebraic theories which have this property remains.
I find this extremely cool because it reduces a question about computability theory - which seems to require a lot of essentially arbitrary choices about how powerful the model of computation should be, etc - to a purely algebraic question.
Hillel Wayne: A Binder Full Of Questions.
Mark Xu: Strong Evidence is Common.
Eigil Fjeldgren Rischel
2021
2
6
https://erischel.com/why-python-is-better-than-haskell/
why-python-is-better-than-haskell
/why-python-is-better-than-haskell/
Why Python Is Better Than Haskell
Also read Hillel Wayne: Why Python Is My Favorite Language.
Tweet
So: I like python a lot. On the other hand, I also really like haskell. I like functional programming. I like putting information into types, and having the types checked at compile-time. I like higher-order function like `map`. By god, I even like monads. I think monadic parser-combinators are just about the coolest thing in the world - especially the fact that you can implement them as a library!
Python is very antithetical to that. It obviously doesn't have static types. It has higher-order functions, but their use isn't super idiomatic. It's not pure. But... I still really like coding in it. In a lot of cases I _strongly_ prefer writing Python to Haskell. Probably the strongest reason for that is the build situation in Haskell, which is extremely stupid. But actually even allowing that, python seems to have a certain ease of use that Haskell doesn't. Why is that? I think the key thing is that _Haskell makes it easy to create new abstractions and regards it as a normal part of coding_.
Wait, what? Weren't we talking about things that make _python_ easier to use? Yes. That's what I mean. The fact that Haskell makes it easy and expected to introduce new abstractions means that, to pick up a new library, I need to understand its abstractions, and probably massage an interface between them and my program. That's super annoying.
When writing in python, there are essentially no abstractions, beyond the bare minimum of procedural "each command modifies variables and does IO and returns a value", along with a very small set of types of data: numbers, strings, maps, lists.
It's not that this makes libraries easy to integrate, it's that it makes them _easy to understand_. To use `requests` to talk to mailgun, here's what I did:
I want to make a post request with some arguments, and get a response back (actually the response doesn't matter that much in this application, but whatever). I build the arguments (strings) and the url (a string), then shove all this into `post` function.
Here's the top google result for "haskell make post request", which I just googled now while writing this post. Okay.. so I need to find a `RequestBody` (what's that?) and use `parseRequest` to turn my url into a request, where I can then insert my body..?
I go to the `http-conduit` docs, where I see this:
Okay, so after staring at this for a bit, it seems like you just put in a bytestring? That's _slightly_ more low-level than I was hoping - to be honest, I don't even know how to format a list of arguments like the above into a http request!
After digging around the docs page for a little while, I find this:
Okay.. that doesn't look too bad. Now we can implement the fragment above in Haskell like this:
parseRequest mgurl
response <- httpLbs request manager
return $ body response]]>
(I actually haven't checked whether this works like I expect it to, so don't be shocked if this code doesn't work as written).
Okay, so in this _very simple_ case, we needed to understand two new types: `Manager` and `Request`. `RequestBody` turned out to be a red herring, although I also looked that one up. I'm going to allow `ByteString` as part of the "standard library", but if I was really writing this code I probably would've had to look up/remember how to convert between string types.
The `Manager` object keeps track of open connections, which is probably not needed for my application. Do I need to close my connections after I'm done or something? I actually make some other http requests earlier in this application - does it matter if I use the same manager?
Why does `parseRequest` use the `IO` monad? (Looks at the docs) okay, it actually uses `MonadThrow`, so I guess I should make a decision about how to handle a parse error - should I just crash with an exception or handle the error more gracefully (the Python version of this script has logging - that's probably what "should" be done).
Is this all very complicated? No. Is it bad design? Also no. `http-conduit` provides stuff like `simpleHttp :: MonadIO m => String -> m ByteString` which lets you do simple requests. I'm gonna assume giving people the option of passing a manager object around is a good idea. Making people be explicit about what request they want instead of just making a best guess based on the arguments passed to `requests.post`.
Once your application gets more complicated, the extra structure imposed by Haskell starts paying dividends - the abstractions begin to simplify things, justifying the upfront cost of understanding them. And the abstractions built by the _user_ - not imported from libraries but custom-built for this application - start becoming worthwhile.
But while you're still at the "fucking around to do something simple" stage, it's all very overkill.
Eigil Fjeldgren Rischel
2021
1
31
https://erischel.com/january-2021-links/
january-2021-links
/january-2021-links/
January 2021 Links
Alvaro de Menard: Are Experts Real?, and the followup, Unjustified True Disbelief.
The former:
> There's a superficial uniformity in the academy. If you visit the physics department and the psychology department of a university they will appear very similar: the people working there have the same titles, they instruct students in the same degrees, and publish similar-looking papers in similar-looking journals.6 The N=59 crew display the exact same shibboleths as the real scientists. This similarity provides cover so that hacks can attain the prestige, without the competence, of academic credentials.
> Despite vastly different levels of rigor, different fields are treated with the ~same seriousness. Electrical engineering is definitely real, and the government bases all sorts of policies on the knowledge of electrical engineers. On the other hand nutrition is pretty much completely fake, yet the government dutifully informs you that you should eat tons of cereal and a couple loaves of bread every day. A USDA bureaucrat can hardly override the Scientists (and really, would you want them to?).
Concavenator: Gods of Salt.
Michael Shulman: The Logic of Space. Especially recommend section 2.1, "On Syntax." See also this discussion of a related point on twitter. "How much information is actually in a universal property" is a fascinating question, one I actually might write a longer post about at some point.
Sarah Constantin: Wrongology 101. Unfortunately seems to suffer from a spot of bad formatting.
Escardó-Simpson: A universal characterization of the closed Euclidean interval. People often say stuff like "analysis is coalgebraic" - the description of the unit interval in this paper in what is essentially finitary terms is probably the purest example I've seen of this.
Interesting thread: Joel David Hamkins On Twitter
Nintil: Longevity FAQ.
SSC IS BACK BABY, now called "Astral Codex Ten". I also recommend going to the website of Scott's new psychiatry practice, Lorien Psychiatry, and reading his writing on psych, especially Ontology of Psychiatric Conditions: Taxometrics. I enjoyed Contra Weyl on Technocracy, but found it a bit weak (I especially feel the "technocracy success stories" weren't very strong). I think this is because both Scott and Weyl are using the term Technocracy in a sort of confused way, although Scott seems to be grappling towards understanding in this post. I think this is an object-level disagreement masquerading as a meta-level disagreement (which is also why all of Weyls actual alternative policy proposals smell so much like more technocracy - technocracy isn't really describing what it is he's against.) I really liked this quote:
> There is no way to perfectly calculate the devastation of a potential pandemic that hasn't happened yet. But once you make even a weak effort, you notice that all the numbers are really really big.
Jeff Kaufman: Bets, Bonds and Kindergarten. I might steal this idea for when I have my own children.
Alice Maz: Alien Intelligences
Philip Wadler: Theorems for free!
Junk Heap Homotopy: The Four Intuitions. (Physical, Computational, Algebraic, Geometric).
Milan Cvitcovic: Things You're Allowed To Do. Put it on Tab Snooze and read it every month.
Eigil Fjeldgren Rischel
2021
1
30
https://erischel.com/notes-from-pfpl/
notes-from-pfpl
/notes-from-pfpl/
Notes from "Practical Foundations for Programming Languages
_Practical Foundations for Programming Languages_ (PFPL), by Robert Harper, is an introduction to the theory of programming languages.
I recently finished reading through it. My read was fairly cursory - I stopped to think about ideas which seemed important or interesting, but I didn't read everything deeply, and I didn't do a lot of exercises.
In this post I'll summarize the things I got from it that seemed really interesting.
I'll be giving my own perspective (much more category-oritented), not trying to stick to the book. This also means I might be mixing in some of my own thoughts that weren't really in the text, without any effort made to distinguish them. Sorry! If there's a dumb thing here don't assume that means the book is dumb.
Eigil Fjeldgren Rischel
2021
1
30
Should you read it?
I don't regret skimming. The book spends a lot of time on concepts which didn't really seem interesting to me - but of course that's highly personal, and I guess it's trying to be somewhat comprehensive, so that's the price you pay.
I might have strongly preferred to read a summary of the interesting ideas (like this one), but it's hard to say how much I would have absorbed - somehow you usually learn more from being a bit immersed in things.
Eigil Fjeldgren Rischel
2021
1
30
Table of contents
(of the book). Reproduced here so you can see if anything strikes your fancy:
1. Judgment and rules
2. Statics and dynamics
3. Total functions
4. Finite data types
5. Types and propositions
6. Infinite data types
7. Variable types
8. Partiality and recursive types
9. Dynamic types
10. Subtyping
11. Dynamic dispatch
12. Control flow
13. Symbolic data
14. Mutable state
15. Parallelism
16. Concurrency and distribution
17. Modularity
18. Equational reasoning
19. Appendices.
My notes won't be following this structure.
Eigil Fjeldgren Rischel
2021
1
30
Induction and coinduction
This is probably the biggest thing I got out of this book.
A lot of this is just my existing learning on this stuff finally crystallizing - so what I've written here may reflect me a lot more than the book.
Given a functor , we can try to solve the equation in at least two ways:
We can take an _initial algebra_ of , or a _terminal coalgebra_ of .
Recall that an algebra is an object with a map .
There's an obvious category of algebras, and it's a theorem that for the initial object , if it exists, the map is an isomorphism.
Dually, a coalgebra is an object with a map , there's a category of coalgebras, and if there is a terminal object , is an isomorphism.
These correspond in some sense to "minimal" and "maximal" solutions of the equation .
Example: Let , .
The initial algbra is the set of naturals, with taking to and to .
The terminal coalgebra is the set of _conaturals_, , with taking to , and every other conatural to (with ).
Let's say you want to construct a map . You can do so by _induction_: specify an -algebra structure on ,
and there'll automatically be a unique homomorphism .
This algebra structure corresponds to a map . This map tells you:
1. is where should go
2. is where should go, if goes to .
In other words, and inductive definition.
Moreover, you can prove properties of such maps using initiality.
Let be an algebra homomorphism, and let's suppose I want to show that for some subset.
Then it suffices to prove that there is an algebra homomorphism with image contained in .
The obvious way to construct this homomorphism is to let be some subset of , contained in .
For to be an algebra, we must have, if , and for .
In other words, must be stable under successor and contain the initial element.
We can in particular do ordinary induction by applying this to the identity map.
Let be the subset of elements satisfying some property.
I want to show that the identity has image contained in , i.e that is the whole thing.
It suffices to show that it contains and is closed under taking successor, i.e to perform induction in the usual sense.
Coinduction is dual to this. It lets you construct maps into the terminal algebra, and prove equality between such maps.
Now maybe let for some set .
The terminal coalgebra is the set of _streams_, functions , with the structure map slicing off the first element.
Given a map , we get a map by terminality.
Moreover, we can use _coinduction_ to show that for some .
Namely, if we can find any coalgebra homomorphism so that ,
then we must have , since we automatically have , with the unique map.
How do we construct such a map? The obvious way is to let the quotient of by some equivalence relation .
For to still be an algebra - in other words, for the map to descend to a map ,
we must have that
1. entails
2. entails
where are the two components of .
The general slogan is that induction lets us understand _maps out of the structure_ - which includes
Eigil Fjeldgren Rischel
2021
1
30
What does "type safety" even mean?
The simplest explanation of this is that type safety is a property of a type system and a "operational semantics",
i.e of a model of program execution. In turn the simplest notion of execution is to give:
- A subset of expressions in the language called "values", which are "done", i.e they represent the results of computation
- A relation on expressions saying "e can evaluate to e' in one execution step".
Then type safety means, if is well-typed of type , either for some , or is a value.
You _don't_ want to say something like "any well-typed program evaluated to a value of that type", because your type system probably doesn't guarantee termination (if it does, you language is not Turing complete).
Obviously you need more sophisticated versions of this to account for more complicated models of program evaluation, programs with side effects, etc, but this is the idea.
Eigil Fjeldgren Rischel
2021
1
30
Quantified and self-referential types
Given a type with a free type variable (like ), we can form the type .
An element of this type should be something that can "play the role" of no matter what is.
For example gives an element of .
The behaviour of elements of this type is basically:
- you can "cast" an element of to , i.e you can "specialize" the value.
- To produce an element of , i have to produce an element that typechecks as , no matter what the value of is (the type variable can't appear in the context).
Similarly you can make sense of "existential types" .
Once you have these, you can define _recursive_ types, too.
Take something like a list of integers: `Data Intlist = Nil | Cons Int Intlist`.
In other words, `Intlist = () + (Int,Intlist)`.
We can type this using universally quantified types as .
This basically means: you give me an and a map , and I give you an , for any type .
The `Nil` constructor is the map which simply returns the given .
The `Cons n l` constructor, for `n` an integer, returns an intlist which, given , feeds it to `l` (returning ),
then computes and returns it.
If you unpack this in your head, you'll see that this is exactly folding. A list is something you can fold over - the provided is the initial accumulator and the is the combination map.
The general formulation of the above is . Essentially, we are picking out the _initial algebra_ of , by saying that this is the universal thing with a map to for each -algebra structure on [^fn:1].
We could also pick out the _terminal coalgebra_, by writing .
This has the dual meaning - given any -coalgbra structure, and a point in the carrier, we obtain a point of the terminal coalgebra.
The terminal coalgebra of is the type of "colists" - possibly infinite lists (of integers).
We see above that to produce a colist, we have to provide an element of some type, as well as a map . means the list is empty, means is the head and the rest of the list corresponds to the pair . This is "iteration", or maybe "cofolding".
Eigil Fjeldgren Rischel
2021
1
30
How a call stack works
This is about how to evaluate programs. Simple "pure functional" languages can "just" be evaluated by successively applying reduction rules. But this is pretty limited - both in performance terms, but also prevents you from doing a few cool things, like _continuations_.
A _call stack_ is basically a list of functions - in the sense of "expressions with a hole in them" - that are waiting for the result of computation, along with a value that's being evaluated, and a "pointer" which tells us whether the next step is to reduce the active value some more, or to return it, i.e plug it into the next function
So eg the stack `a;b;c > e` means "we're in the process of evaluating `e`. Once that's done, evaluate `c e`, then `b (c e)` and so on". On the other hand `a;b;c < e` means "`e` is a value that's done evaluating, pass it to `c`, then `b`, then `a`.
Then you need to make up some rules for how to handle these things - which order to evaluate expressions and so on.
To augment this language with continuations, you can add expressions like `callcc : (Cont(a) -> a) -> a` and `throw : a -> Cont(a) -> b`, where the meaning of `callcc` is "run this function with a "continuation" pointing the current state - if the continuation is ever "used" by `throw`, just immediately return the value `a` that was supplied".
Then the dynamics of this stuff is:
- `s > callcc f` evaluates to `s > f [s]`, where `[s] : Cont a` is a representation of the current stack, and
- `s' > throw a [s]` evaluates to `s > a`, i.e throwing restores the stack from the time the continuation was created.
Eigil Fjeldgren Rischel
2021
1
30
Parametricity
You might have heard about "free theorems", from Wadler's paper Theorems for Free!.
Parametricity is the technical term for those things.
The basic idea is that a term of polymorphic type has certain properties _purely by virtue of its type_.
This is the so-called "free theorem" associated to that type.
The basic idea behind parametricity is to associate a relation to every type:
- Elements of "primitive" types like and such are equivalent if they're equal (this is just a relation on the type itself)
- Elements of the product type are equivalent if they have equivalent coordinates (according to the relations associated to and )
- Elements of function types are equivalent if they map equivalent terms to equivalent terms, i.e if implies .
- Let be a "type operator" - we can interpret it as an operation on _relations_ - in particular, it can act on relations which are not defined on a single set. The relation is the relation on the set defined by if for all relations .
The statement of parametricity is then that for all (well-typed) terms of closed type.
For example, parametricity for means that for any relation , we have if .
Specializing eg to if and only if for some fixed element (taking maybe ), we see that must be the identity. The precise version of this is more complicated, because you need to deal with the possibility that there may be free type variables around, and also the fact that a type is not really a set (so you need to work with relations on _terms_ - this is what the book does - or use domain theory - this is what Wadler does).
[^fn:1]: Although the quantified definition is more general, because it works even when is not a functor in , but just some type that depends on
Eigil Fjeldgren Rischel
2021
1
21
https://erischel.com/numbers/
numbers
/numbers/
Where numbers come from
Alternative title: Wolves hate him!! Shepherd compares the size of large sets with this one easy trick!
Previously: Recognizing Numbers
Let's do a thought experiment.
I place an empty box in front of you. Then, while you're watching, I put these objects into the box:
Then I remove these things from the box:
You're surprised! Why? Because what I took out is not a subset of what I put in. A new apple appeared.
You can do this experiment with animals, and small children of various ages, and monitor them carefully to see if they seem surprised. You can also try *larger collections of apples*, to see how large a collection of apples they can keep track of.
Once children are old enough to talk, you can make the experiment more reliable by simply asking them if the box is empty. But of course, there's a small window of interestingness here - children beyond a certain age rapidly get extremely good at this problem, and from a certain point humans basically never fail at this task, unless the pile of apples gets extremely large. This does not surprise you at all.
The following picture switches back and forth between two collections of apples. Can you tell whether they're the same size "in one go" - without letting it switch back and forth more than once?
This, it turns out, is actually very hard. Even grown human brains don't come hardwired with an arbitrarily powerful "compare the size of two collections" module. You can compare the *visual* size, which can give you the answer if the relative difference in size is moderately large. But in a case like the above, it's very hard to tell the size of those two collections apart.
Here's a simple piece of technology for comparing the size of two collections: pair of the elements one after the other. If the collections are exhausted at the same time, they're the same size. If not, whichever has elements left at the end is bigger.
Of course, this won't work if the collections are not available *at the same time* to be compared. This could be for a contrived reason like above, the image flashing back and forth. Or it could be for a practical reason - a shepherd wants to compare the set of sheep in the pen when he opens the gate in the morning to the set of sheep in the pen before he closes the gate at night.
So humans, so long ago that the origins have been completely forgotten, but certainly more than 20.000 years ago, came up with an ingenious technology to solve this problem:
I will describe it for you now.
We invented a *reference set* of every possible size (infinities had not been invented yet). There are many such families of reference sets, but here is the one you are probably familiar with:
Before I let out the sheep in the morning, I *identify the reference set with the same size as the collection of sheep*. I do this by the procedure used above - I match up sheep with elements of the reference sets until I run out of sheep. "1,2,3,4,5,6,7,8,9,10,11". Now I know that the collection of sheep has the same size as the collection
. This is called *counting*.
In the evening, I compare that collection with the collection of sheep that came back - if they're not the same size, I know about the discrepancy.
It's important to emphasize that numbers are really a *technology* - it had to be invented. We know this because there exist communities without this technology. You've probably heard about languages without a name for numbers above 2 - "one, two, many". Well, that's more or less real. The most famous are the Pirahã of the Amazon rainforest.
Their language has two words "hói" "hoí" (distinguished by tone) - originally taken to mean "one" and "two", but now believed to probably mean something like "small quantity" and "larger quantity". These are the closest thing to numerals in their language.
Experiments like the one I described at the beginning have been put to them[^2] - even adult humans usually begin to fail at this task even when the number of objects is as low as four or five. They *don't* fail at this task when simply asked to match the number of object placed in a line of the table. They understand what it means for two sets to be *in bijection*, but they lack the technology of numbers to keep track of this information. The Pirahã are quite capable of hunting, gathering, cultivating manioc, crafting bows and arrows, building huts, and generally surviving in the jungle. They're not *stupid*. But they really, truly, do not know how to count.[^3]
[^3]: Actually, it seems some meddlesome people have started teaching the Pirahã Portugese, including numerals, and basic mathematics. So the world may be about to lose one of the few examples of numberless peoplmay be about to lose one of the few examples of numberless people..
The main trick here is *abstraction*. We remove all the details of the individual sheep and remember *only* the "number" - the *size* of the collection, its ability to count other things, be in bijection with other things. A mathematician might say "the bijection class of the set", if they did not have the word "number".
The second trick, also important, is *reification*. You can see how much I fumble for words above, trying to describe the concept "the number of elements in a set" without using the word "number". This is not a quantity you can put inside your brain. So we choose a *simple representative*. Whoever made the Ishango bone, pictured above, choose a set of marks on a bone to represent the bijection class. This is convenient because you can just keep the set of marks, i.e the bone, with you until you need the number again (unlike the set of sheep, which you have to let out to graze, that's the whole point). Another implementation is by creating a set of *words*. The set , or , is a handy set of a given size.
We name this set after its largest element - "three" - and to reconstruct the set from the name, you only need to recall the order of the special size-words[^1]
[^1]: In mathematics, we might define to be the set instead - this has the advantage that the definition is not self-referential, and maybe technically convenient for other reasons. But most people count starting from 1, not 0.
[^2]: See Number as a cognitive technology: Evidence from Pirahã language and cognition. Concretely, the subjects were asked to match the number of objects placed by the experimenter. In one experiment, the experimenter simply put objects down in a line on the table. In another, the objects were dropped one after the other into an opaque container. The subject then had to place the same number of objects on their side of the table. There were a few different versions of this. Maybe it's important to note here that this study was not exactly high-n, and communication with the subjects was unreliable for obvious reasons. There were a few failures even on the "easy" versions of the tasks, so perhaps it's not entirely clear how much of the results should be put down to the subjects having trouble representing cardinalities in their head, and how much should be put down to different versions of the task being harder to understand, or even simply deciding to mess with the experimenters.
(Thanks to John for inspiring me to write this).
Eigil Fjeldgren Rischel
2020
12
31
https://erischel.com/december-2020-links/
december-2020-links
/december-2020-links/
December 2020 Links
A list of some of the things I found interesting in December.
Low-Tech Magazine: "Low-tech Magazine questions the blind belief in technological progress, and talks about the potential of past and often forgotten knowledge and technologies when it comes to designing a sustainable society. Interesting possibilities arise when you combine old technology with new knowledge and new materials, or when you apply old concepts and traditional knowledge to modern technology". Sample articles: The Curse of the Modern Office, Well-tended fires outperform modern cooking stoves, Why we need a speed limit for the internet.
Evangelia Aleiferi: Cartesian Double Categories with an Emphasis on Characterizing Spans.
Owen Lynch: Haskell's Children (Owen's whole blog is worth reading).
Tobias Fritz has put up some slides discussing our new preprint about comparison of experiments (with Paolo Perrone and Tomás Gonda).
I've been rereading Bayesian Updates Compose Optically by Toby St Clere Smithe.
Michael Nielsen: Principles of Effective Research.
Applied Divinity Studies: Isolated Demands for Rigor in New Optimism. "More likely, solar power has been making great strides on a pretty consistent basis for decades, and the only recent break is in how high-status it is to say that out loud."
Jules Hedges: Compositional Game Theory Reading List
Jason Collins: Principles for the application of Human Intelligence
George on LessWrong: Machine Learning May Be Fundamentally Unexplainable. "When we say that we “understand” physics what we really mean is that there are a few dozen of thousands of blokes that spent half their lives turning their brains into hyper-optimized physics-thinking machines and they assure us that they “understand” it."
Amanda Askell: In AI Ethics, "Bad" Isn't Good Enough.
A map of properties of logical theories: <https://forkinganddividing.com/>
r/math: Most overpowered theorems in math?
Eigil Fjeldgren Rischel
2020
12
9
https://erischel.com/galois-connections-and-nullstellensatzen/
galois-connections-and-nullstellensatzen
/galois-connections-and-nullstellensatzen/
Galois Connections and Nullstellensatzen
(The idea for this post is due to this tweet by @sarah_zrf)
Eigil Fjeldgren Rischel
2020
12
9
Hilbert's Nullstellensatz
Consider the ring of complex polynomials in variables, .
The elements of thing ring can be viewed as _functions_ .
Given a polynomial , we can think of it as an _equation_ in variables - a solution to the equation is a tuple
so that . Let be the set of solutions.
Given a _set_ of polynomials , let
be the set of solutions to the _system_ of equations given by .
This gives a mapping from subsets to subsets .
We can also go the other way - given such an , let be the set of such that for all .
(We are now writing for brevity).
Now and enjoy the following very special properties:
- They are both _order-reversing_ - if then , and the same for .
- and .
This means that and forms a _Galois connection_ (nlab) - a dual adjunction between two posets.
This implies a bunch of interesting things. One of the most important is that the mapping is a _closure operator_, meaning that in addition to the above inequality, . In fact for anything of the form , we have .
This Galois connection is _the_ key to algebraic geometry. It allows us to analyse algebraic things (subsets of a ring) in geometric terms (subsets of a space). So it would be really good if we could understand this thing. In particular, what is the closure operator ?
It's pretty clear that must be an _ideal_ - if for all , then too (and if , then ). So an obvious guess would be that , the ideal generated by .
But this turns out not to be true - there's one more thing we need to close off under to make this work.
Namely, if , then also . So if , then (and similarly for ).
is a _radical ideal_ - specifically, it is the radical of the ideal generated by . This statement is _Hilbert's Nullstellensatz_ ("zero locus theorem", see wikipedia, nlab)
Eigil Fjeldgren Rischel
2020
12
9
Gödel's Nullstellensatz
Let be a first-order language, i.e a collection of _function symbols_ , each with a specified arity, and /relation symbols , also each with a specified arity[^fn:1].
An -structure is a set equipped with a function for each , where is the arity of , and a subset for each , where is again the arity.
A _first-order formula in _ is a formula built up out of the function symbols, variables, and the connectives .
A model satisfies the formula if the formula is true when interpreted in the usual way for the model.
A _theory_ is a set of formulas. Given a theory , let be the set of models that satisfy each formula in .
Given a set of models , let be the set of formulas that are satisfied by each model.
Then it's not hard to see that and form a Galois connection between sets of formulas (theories), and sets of models.
What is the closure operator on theories induced by this Galois connection?
In other words: Start with a set of formulas. These formulas pin down a set of models, namely the models that satisfy all those formulas. Now almost certainly some more formulas are going to happen to be true for all these models.
For example, if is a formula in , then also is going to be true for every model - so .
In general, any formula which is _provable_ using the normal rules of logic from formulas in is going to be in . This is because the normal rules of logic are _sound_ - if you can prove something from true premises, it's true.
And in fact, this suffices! To be precise, consists _exactly_ of those formulas that are provable using the normal deduction rules of first-order logic from . This is a celebrated theorem of Gödel (nlab, wikipedia). Since it has the same formal structure as Hilbert's Nullstellensatz - characterizing the closure operator induced by a Galois connection - we might call it _Gödel's Nullstellensatz_. (It is usually called the completeness theorem).
Eigil Fjeldgren Rischel
2020
12
9
Quillen's Nullstellensatz
Fix a category . Let be morphisms.
We say that _ has the left lifting property with respect to _, and that has the right lifting property with respect to , if, for each diagram of this form, if the outer square commutes, there exists a dashed arrow making the triangles commute as well:
Given a class of morphisms in , let be the class of morphism with the left lifting property with respect to _all_ morphisms , analogously .
It is again not hard to see that and form a Galois connection from the collection of subclasses of morphisms in to itself. What is the closure operator ? This is probably a bit harder to motivate than the previous examples, but understanding it is important in model category theory.
As before, we can try to find some constructions that is certainly closed under:
- Pushouts, in the sense that if is in , and is any morphism, the induced is also in there.
- Retracts: given a commutative diagram
where the horizontal composites are identities, if is in the class, so is
- Transfinite composition: given a sequence with each map in the class, the maps to the colimit are also in the class (here this diagram can be indexed by any ordinal).
Each of these proofs is essentially by a diagram chase.
A class closed under all these constructions is called _saturated_. Then we have _Quillen's Nullstellensatz_ (which is confusingly usually called the _Small object argument_): If is locally presentable, and is a small set,
then is the smallest saturated class containing .
I guess you can actually dispense with the function symbols if you want
Eigil Fjeldgren Rischel
2020
11
22
https://erischel.com/demystifying-the-second-law-of-thermodynamics/
demystifying-the-second-law-of-thermodynamics
/demystifying-the-second-law-of-thermodynamics/
Demystifying the second law of thermodynamics
Thermodynamics is really weird. Most people have probably encountered a bad explanation of the basics at some point in school, but probably don't remember more than
Energy is conserved
Entropy increases
There's something called the ideal gas law/ideal gas equation.
Energy conservation is not very mysterious. Apart from some weirdness around defining energy in general, it's just a thing you can prove from whatever laws of motion you're using.
But _entropy_ is very weird. You've heard that it measures "disorder" in some vague sense.
Maybe you've heard that it's connected to the Shannon entropy of a probability distribution .
Probably the weirdest thing about it is the law it obeys: It's not conserved, but rather it _increases_ with time.
This is more or less the only law like that in physics.
It gets even weirder when you consider that at least classical Newtonian physics is _time-symmetric_.
Roughly speaking, this means if you have a movie of things interacting under the laws of Newton, and you play it backwards, they're still obeying the laws of Newton. An orbiting moon just looks like it's orbiting in the other direction, which is perfectly consistent. A stone which is falling towards earth and accelerating looks like it's flying away from earth and decelerating - exactly as gravity is supposed to do.
But if there's some "entropy" quality out there that only increases, then that's obviously impossible! When you played the movie backwards, you'd be able to tell that entropy was decreasing, and if entropy always increases, some law is being violated.
So what, is entropy some artefact of quantum mechanics? No, as it turns out. Entropy is an artefact of the fact that you can't measure all the particles in the universe at once. And the fact that it seems to always increase is a consequence of the fact that matter is stable at large scales.
The points in this post are largely from E.T. Jaynes' Macroscopic Prediction.
Eigil Fjeldgren Rischel
2020
11
22
A proof that entropy doesn't always increase
Let be the set of states of some physical system. Here I will assume that there is a finite number of states and time advances in discrete steps - there is some function which steps time forward one step. We assume that these dynamics are time-reversible in the weak sense that is a bijection - every state is the future of exactly one "past" state.
Let be some function. Assume - in other words, can never decrease.
Then is constant, i.e .
Proof: Assume for contradiction for some . Since is finite, let be the sum of over all states. Then clearly , since just ranges over all the s.
But on the other hand, we have for all , and in at least one case. So we must have - contradiction.
This proof can be generalized to the continuous time and space case without too much trouble, for the types of dynamics that actually show up in physics (using Liouville's Theorem). The proof above still requires a _bounded_ phase volume (corresponding to the finiteness of ). To generalize to other situations we need some more assumptions - the easiest thing is to assume that the dynamics are time-reversible in a stronger sense, and that this is compatible with the entropy in some way.
(You can find easy counterexamples in general, e.g. if and the dynamics are , then obviously we really do have that is increasing. Nothing to do about that.)
Anyways the bounded/finite versions of the theorems do hold for a toy thermodynamic system like particles in a (finite) box - here the phase volume really is bounded.
Eigil Fjeldgren Rischel
2020
11
22
The true meaning of entropy
Okay, so what the hell is going on? Did your high school physics textbook lie to you about this?
Well, yes. But you're probably never going to observe entropy going down in your life, so you can maybe rest easy.
Let be the physical system under consideration again. But suppose now that we can't observe , but only some "high-level description . Maybe is the total microscopic state of every particle in a cloud of gas - their position and momentum - while is just the average energy of the particles (roughly corresponding to the temperature).
is called a _microstate_ and is called a _macrostate_.
Then the _entropy_ of is - the logarithm of the number of microstates where . We say these are the microstates that _realize_ the macrostate .
The connection with Shannon entropy is now that this is exactly the Shannon entropy of the uniform distribution over . This is the distribution you should have over microstates if you know nothing except the microstate.
In other words, the entropy measures your uncertainty about the microstate given that you know nothing except the macrostate.
There are more sophisticated versions of this definition in general, to account for the fact that
- In general, your microstates are probably sets of real numbers, and there are probably infinitely many compatible with the macrostate, so we need a notion of "continuous entropy" (usually called differential entropy, I think)
- Your measurement of the macrostate is probably not that certain (but this turns out to matter surprisingly little for thermodynamic systems),
but this is the basic gist.
Eigil Fjeldgren Rischel
2020
11
22
Why entropy usually goes up
Okay, so why does entropy go up?
_Because there are more high-entropy states than low-entropy states_. That's what entropy _means_.
If you don't know anything about what's gonna happen to (in reality, you usually understand the dynamics themselves, but have absolutely no information about except the macrostate), it's more likely that it will transfer to a macrostate with a higher number of representatives than to one with a low number of representatives.
This also lets us defuse our paradox from above. In reality, entropy doesn't go down for literally every microstate .
It's not true that S(p(x))]]> for all - I proved that impossible above.
What _can_ be true is this: given a certain macrostate, it's more probable that entropy increases than that it decreases.
We can consider an extreme example where we have two macrostates and , corresponding to low and high entropy.
Clearly the number of low-entropy states that go to a high-entropy state is exactly the same as the number of high-entropy states that go to a low-entropy state. That's combinatorics.
But the _fraction_ of low-entropy states that go to high-entropy is then necessarily larger than the fraction of high-entropy states that go to low-entropy states.
In other words, P(L(x_{t+1})|H(x_t))]]>
Eigil Fjeldgren Rischel
2020
11
22
Why entropy (almost) always goes up
Okay, but that's a lot weaker than "entropy always increases"! How do we get from here to there?
I could say some handwavy stuff here about how the properties of thermodynamic systems mean that the differences in the number of representatives between high-entropy and low-entropy states are massive - and that means the right-hand probability above can't possibly be non-neglible. And that in general this works out so that entropy is almost guaranteed to increase.
But that's very unsatisfying. It just happened to work out that way? I have a much more satisfying answer: entropy almost always increases because matter is stable at large scales.
Wait, what? What does that mean?
By "matter is stable at large scales", I mean that the macroscopic behaviour of matter is predictable only from macroscopic observations. When a bricklayer builds a house, they don't first go over them with a microscope to make sure the microstate of the brick isn't going to surprise us later. And as long as we know the temperature and pressure of a gas, we can pretty much predict what will happen if we compress it with a piston.
What this means is that, if , then _with extremely high probability_, . It might not be literally certain, but it's sure enough.
Now, let's say we're in the macrostate . Then there is some macrostate which is _extremely likely_ to be the next one. For very nearly all so that , we have .
But this means that must have at least that many microstates representing it, since is a bijection.
So the entropy of can at most be a _tiny_ bit smaller than the entropy of - this difference would be as tiny as the fraction of with , so we can ignore it.
So unless something super unlikely happens and , entropy goes up.
By the way, this also explains what goes wrong with time-reversibility, and why in reality, you can easily tell that a video is going backwards. The "highly probably dynamics" , which takes each macrostate the the most probable next state, don't have to be time-reversible. For instance, let's return to the two-macrostate system above.
Suppose that with 100% certainty, low-entropy states become high-entropy.
Let there be low-entropy states and high-entropy states.
Then, just because is a bijection, there must be high-entropy states that become low-entropy.
Now if , then practically all high-entropy states go to other high-entropy states.
So but .
Of course in reality, if you start with a low-entropy state and watch this unfold for a _really_ long time, you'll eventually see it become a low-entropy state again. It's just extremely unlikely to happen in a short amount of time.
Eigil Fjeldgren Rischel
2020
11
22
Entropy is not exactly your uncertainty about the microstate
The entropy of a given macrostate is the uncertainty about the microstate _of an observer who knows only the macrostate_.
In general, you have more information than this.
For example, if the system starts in a low-entropy state, and you let it evolve into a high-entropy state, you know that the system is in one of the very small number of high-entropy states which come from low-entropy states!
But since you can only interact with the system on macroscales, this information won't be useful.
Eigil Fjeldgren Rischel
2020
11
18
https://erischel.com/a-response-to-maudlin-on-credence-and-chance/
a-response-to-maudlin-on-credence-and-chance
/a-response-to-maudlin-on-credence-and-chance/
A response to Maudlin on credence and chance
Credence - and chance - without numbers (and with the Euclidean property) is a philosophy paper by Tim Maudlin.
In it, Maudlin discusses the closely related notions of _credence_, the subjective likelyhood that a specific agent associates to some outcome, and _chance_, the objective likelyhood that the event happens. He argues that
The traditional approach of measuring these outcomes with numbers is wrong, but
this shouldn't trouble us too much, because we can do a lot without them.
As it happens, I have my own serious reservations about the traditional measure-theoretic formulation of probability (due to Kolmogoro). By I still think this paper is more or less terrible. Maudlin displays a Wikipedia-level understanding of the paradoxes surrounding infinity in probability theory. His proposed solution amounts to enumerating a list of properties that an agent's system of credence should satisfy, and arguing that these axioms suffice for the things we usually want to do with "subjective degree of belief". This is actually fine such as it is, but his description of this is also seriously lacking.
Eigil Fjeldgren Rischel
2020
11
18
Infinity in probability theory
There's the following very classical paradox in probability theory:
Suppose you toss a dart at a dartboard. For each point on the dartboard, we can ask for the probability that the dart hits that exact point . Suppose we have no relevant information about the tossing process, so that each point seems equally likely.
What is the probability ? Well, it can't be more than zero, because then the total probability, the probability that the dart hits any point, would be, not just greater than , but infinite! This is because there are infinitely many points on the dartboard, and any positive number added to itself infinitely many times is infinite.
But hold on, it also can't be , because that means by a similar argument that the probability that the dart hits any point on the dartboard is also - but it must hit _some_ point.
First I'll note that this problem actually is _not_ entirely resolved by measure theory.
Measure theory allows for a uniform probability measure on continuous spaces like the interval (by, essentially, not allowing us to add probabilities up in every case. More on this later). But if we just suppose a countable dartboard, we're left with the same problem.
But, hang on, does a dartboard really have an infinite number of points? And what does it mean for the dart to hit a point?
Certainly the point of the dart _itself_ has a non-infinitesimal area, so that if we partition the dartboard into "points" of that size, there will be a finite number, each of which we can assign some positive probability without issue. In general, any area on the dartboard, no matter how small, has a positive probability of being struck by the dart, and this remains true no matter how small we make the dart, as well.
In other words, this paradox hinges on an infinitesimal dart, as well as an infinitesimally accurate measuring tape with which to measure the coordinates, which presumably has infinitely long numerals on it (presumably the person assigning these credences has an infinitely large brain as well, to be able to hold in their minds the coordinates pointing out a specific point on the dartboard to infinite precision).
Any mathematician knows that you have to handle a situation like this _very carefully_. You cannot expect to apply your intuition directly and get consistent results. I am actually kind of surprised to find serious philosophers making this kind of argument in 2020. This is on par with saying that Zeno's paradox proves you can't use numbers to measure space (or time).
To be fair to Maudlin here, I do think he has a point: there do seem to be situations where you should be indifferent between a (countably) infinite number of outcomes, and probability theory won't let you do that. Dartboards is just a bad example.
Maudlin's shaky relationship with infinity shows up again in his discussion of the classical Kolmogorov approach to probability. He cites the definition of a _finitely additive_ probability measure, and not the -additive (or countably additive) version used by every probability theorist and statistician in the world. Then he criticizes it for only being finitely additive!
He also critizises the use of a -algebra, arguing that it seems irrational to exclude certain sets from consideration.
I think Maudlin needs to spend a small amount of time understanding non-measurable sets before arguing they're stupid.
For one thing, a nonmeasurable subset of (such as we might ask "will the dart hit a point in this subset or not" of), is _extremely weird_. It's consistent with ZF minus Choice that there exist no such sets.
Certainly such sets always contain regions of "infinite complexity", in the sense that there are (non-infinitesimal) regions of the dartboard so that, if the dart hits there, no finite amount of measurement will suffice to determine whether it is in the set or out of it. Indeed, it's not too far off the mark to think of the -algebra as denoting the "determinable" events.
The fact that a probability measure is only _countably_ additive is still a bit weird. You can make philosophical arguments for this choice: To check whether , we can check whether for each - and this is guaranteed to give you an answer in a finite amount of time, but only if you're taking a finite union. There are reasonable counterarguments to this, but Maudlin doesn't engage with this topic at all - it really seems as if he doesn't realize that mathematicians have been taking infinite disjunctions of events all this time[^fn:1].
Eigil Fjeldgren Rischel
2020
11
18
Maudlin's solution
Maudlin's proposed replacement for numbers is essentially the following idea: it's not required that a person's credences be represented by some definite "thing" - a number or another mathematical object. Instead we should ask about the _structure_ on the system of credences, and what rules this should follow for any "rational" person.
This is actually a very good idea! Structuralism! Defining things extrinsically rather than intrinsically! I like this approach.
Let's go over the basic principles that Maudlin comes up with. The paper is short on rigor, but since it's a philosophy paper and not a math paper, I won't hold that against it.
- The set of "credences" should have a partial order[^fn:2].
- The map which assigns an event its credence should be order-preserving - in other words, if event entails the event , then .
- There should be a least credence and a greatest credence , corresponding respectively to the credence in an event which is certain not to happen or certain to happen (say, a contradiction and a tautology).
- The map should have the following property: if ,
then . This is the so-called "Euclidean property" that Maudlin cannot stop going on about: "the whole is greater than the part", or in this case, if entails , and it's possible that but not (i.e. does not entail ), then our credence in must be _strictly greater_ than our credence in .
We can define a partial addition operation on credences, denoted in the paper,
by setting whenever .
It's not exactly clear in the paper whether we extend this using the equality on credences - for example, whether we can assign a value to by finding some other event so that , and setting . This seems a harmless enough extension (if we really regard credences as _equal_, and not just equivalent in some sense, it seems inevitable), but Maudlin doesn't really spell it out. Nevertheless the addition is still _partial_, since, for example, when rolling a fair dice, we can't add our credence in the event "The result will be one of 1,2,3,4" to itself, since there is no mutually exclusive event which is equally likely.
This is not really a bug - indeed, while you can add probabilities and get a number greater than 1, you can't really treat that number _as a probability_.
So far, Maudlin's approach differs from the Kolmogorovian approach chiefly in that he lets the poset of credences be any ordered set, not just numbers, and that he doesn't require the order on credences to be _total_ - we can have no opinion about the relative plausibility of two events without declaring them _equally_ plausible. This is reasonable enough.
Probably the weirdest thing here is that the "Euclidean property" is _also satisfied by Kolmogorov probability!_
The reason that adding a point to a set doesn't change the probability that the dart will land there is simply that it's impossible that the dart will land at that precise point. This is obviously counterintuitive (and there are ways of defending it, and counterarguments to those, and so on - but Maudlin does not engage with this at all), but that is the classical approach. It does not seem to violate the Euclidean property at all.
Having the addition in hand, we can actually reformulate the Euclidean property simply as " is cancellative", i.e if then . This means that , and if this equals then , since is the unit of addition.
Now Maudlin wants to talk about _relations_ between credences. First of all I have to go on a rant about his whole discussion about ratios. Mathematics has come a long way since Euclid, and citing him like you actually believe his axioms provide a sound formal basis of geometry is not gonna convince anyone that you know what you're talking about.
Maudlin goes oddly pedantic and says that the ratio of a circle's circumference to its diameter is not , but rather, the ratio of the number to the number , and this is in general what people mean when they identify a ratio with a number. I'm glad to hear that Maudlin has found a solution to the long-standing problem of the identity of mathematical objects (presumably resolving it in favor of a _very_ strong form of Platonism?), but I couldn't find this in the bibliography of the paper or any of his other work. Until I do, I have to insist that "loose talk" is when you insist that two mathematical objects identified under isomorphism cannot possibly be called equal, not the other way around.
Okay, rant done. We can define the ratios between credences by addition. For example, we can say that credence is twice as big as credence if . Maudlin, quoting Euclid: "Magnitudes are said to have a ratio to one another which can, when multiplied, exceed one another". There is however a serious problem with this: the partiality of addition means that you can't _compare_ these ratios, and many pairs of credences have no ratio to one another, even though Maudlin insists that they do.
Let's pull back here a bit. Suppose I have an ordered (commutative) monoid .
Then I can say that have a ratio to each other if there exist so that , and . If this is true, we can actually obtain a unique real-numbered ratio "" as the supremum of all so that .[^fn:3] Here I'm using to denote the -fold sum .
Now, this _doesn't_ work for credences - even in situations where it feels like it should[^fn:4]. The increasingly accurate rational approximations to the "true" real ratio requires longer and longer chains of addition, but this is generally impossible to define for credences. We can define what it means for to be an integer - namely that . We can also _sometimes_ define what it means for to be a rational number - for example if , we can say .
But if e.g. , there's no way to make this work. And, crucially, we can't usually compare ratios with each other either - otherwise, we might have argued that expecting number representations for each ratio is too much.
As it stands, by Maudlin's (or rather, Euclid's) definition of ratio, there is no ratio between and any event with "probability greater than " (speaking informally - of course credences aren't probabilities and so on), in contrast with how Maudlin uses it.
This is an area where the paper could really have benefited from some rigor. It's not clear whether Maudlin means to tack on "ratios" as a primitive notion - so that, in addition to having a "more likely than" relation on credences, we have ratios between certain pairs of credences, which can be compared.
All these issues could be solved _very easily_, by imposing some extra structure on the model. For instance we could add "formal sums" of credences which don't have a sum, and say that the statement makes sense even when is not defined as a credence.
Eigil Fjeldgren Rischel
2020
11
18
Closing
As I mentioned above, I actually, genuinely think it's a good idea to think about replacements for classical probability theory. In some sense my own work on Markov categories is a version of this, a sort of "synthetic probability theory". Much like Maudlin, Markov categories start by asking "what structure do we actually need to do the job of probabilities?". They go in a very different direction, not least because we actually care about the job that probabilities do for people in the real world, but the spirit is actually similar.
But stress-testing your theory by applying it to situations involving infinity - without working rigorously with infinity _at all_ - is bound to get you into trouble. Frankly, if the only problem your theory solves is that it makes sense of infinite conjunctions, it's basically useless. Infinite conjunctions only come up in abstrac mathematical models, and trust me, those guys aren't really looking for a new theory. In the real world, it's very hard to write down infinitely many statements and think about your credence in the statement "one of these is true".
The mathematical idea in this paper - which, again, I actually don't think is a stupid idea - could have fit in a couple of pagers, including a fair amount of exposition and the easy fixes to the problem mentioned above. The philosophical work here seems mainly to be a takedown of traditional probability, which falls woefully flat.
[^fn:1]: A less elegant reason for asking for countable composition but not general infinitary composition is that it leads to a useful, theoretically convenient theory, but still allows for measures like the uniform measure on the interval. This is just an argument from practicality - we take these axioms instead of some others because they make the calculations work out - so I won't hold it against Maudlin if he doesn't take this argument very seriously
[^fn:2]: Maudlin seems to not know this term, which probably could have cut several pages from his exposition
[^fn:3]: Ironically, in the usual construction of real numbers, they are limits of ratios (between integers), so one could make a convincing argument that a real number really is literally a ratio.
[^fn:4]: Credences can be "infintesimal, in the sense that we might have for ALL . That's not what I'm talking about."
Eigil Fjeldgren Rischel
2020
11
10
https://erischel.com/remarkable-2-review/
remarkable-2-review
/remarkable-2-review/
reMarkable 2 review
I recently got a reMarkable 2. I've had it for one week. This is my review of it so far.
Eigil Fjeldgren Rischel
2020
11
10
TLDR
- It's very very good and I'm happy I bought one. If your relationship with working "on things" is like mine, I recommend it.
- It's expensive and you can get an iPad + accessories + Apple Pencil for the same price, which may be better
- There are some annoyances
Would I buy the reMarkable again? That is, if it was struck by a meteor tomorrow, would I buy a new one? Yes (but see my notes on other options). In my opinion, these devices are now good enough that it's worth paying a premium to get the "paper experience".
I would not recommend saving up for the reMarkable - that is, if your disposable income is small and you'd need to make an effort over a time period to find the money, it's probably not worth it.
If you own a laptop and are considering buying a tablet, which you anticipate using largely for reading and writing, the reMarkable is recommendable. If you're in the market for a tablet to use also for laptop tasks, it's not up to the job.
Eigil Fjeldgren Rischel
2020
11
10
Paper and Me
My fiancé is one of those stationery nerds. She own several different fountain pens and a metric boatload of ballpoint pens, gel pens, highlighters, markers, and what have you. She buys a specific brand of Japanese calendar (Hobonichi) each year. She has had the same color-coding system for her notes since we started university [^fn:1]. She meticulously pagemarks and sorts all her notes into plastic sleeves and binders.
I am not like my fiancé. My workflow for most of my first five years at university consisted of the following:
- Keep a stack of papers and ballpoint pens in my backpack (along with the books I'm using and my laptop and so on).
- When you sit down to work somewhere, get laptop, stack of papers, and a ballpoint pen out.
- When you need to think, locate a blank spot on a piece of paper in front of you and write there.
- If there's no space, go through the papers until you find some space.
- When you stop working, gather all the papers into your backpack.
- Periodically go through your papers and throw out any that have writing all over them
- Periodically add a fresh dump of blank papers to your backpack.
The two points I want to make here is:
- I am a very disorganized person
- I fucking love paper
Paper is awesome. I am long paper. Whoever you are, you should be using more paper.
There's absolutely no reason to strain your head to the breaking point, keeping a bunch of things inside it, instead of putting them on paper. Actually, if you take one recommendation from this review, just make sure to keep paper and pen wherever you work and use it all the time. Go to these links to buy some _right now_, they're cheap (but you probably have this stuff already - use it!). See also Paper Trauma. I also invite you to consider buying a big notebook, but I haven't tried that yet myself.
But paper has some limitations:
- You sometimes run out
- If you want to write your notes about something you're reading on paper, the best place is the thing you're reading - but this requires that it be on paper (instead of a computer). This means you have to buy a book (these have a tendency to pile up rapidly, if it even is a book you can buy) or print it out (this tends to be messy, and you might not have a printer. Back at the university of Copenhagen, I used to print out a lot of stuff and spiral-back it, but I can't do this any more).
- Sometimes you lose your old paper where you wrote something important (usually, because you didn't know at the time it would be important).
- It's easy to make a mess by creating a huge stack of paper. If you're like me, "clean up your big pile of papers" can sometimes become a bit of an "ugh" task that you can't get around to and which sucks up a lot of energy.
Enter the reMarkable 2. A "next-generation paper tablet", it promises you that you can "Replace your notebooks and printed documents with the only tablet that feels like paper". Naturally, I was intrigued.
Eigil Fjeldgren Rischel
2020
11
10
General experience and thoughts
The writing experience is _very very good_. It doesn't quite feel like paper - it's not _that_ close - but apart from actually writing on paper it's the best I've tried (having, admittedly, not tried that many). It's leagues ahead of writing on a "normal" tablet using a stylus.
Drawing on the reMarkable has the following advantages over paper:
- You can erase things, perfectly
- You can undo/redo strokes
- You can zoom
- You can select and move things
- There's layers, like in image editing software.
The surface feels a bit too smooth - it doesn't have the same friction as dragging a pen over paper. It's still better than a normal screen (like a iPad) though. You can tell they added some texture to make this more lifelike, but it's not the same.
Not surprisingly, it's a bit awkward to hold the tablet with one hand and write with the other. The holding hand doesn't provide quite enough support. You really need to support it with a table or your knees or something.
Probably the biggest thing I would like is a bigger screen.
Eigil Fjeldgren Rischel
2020
11
10
Notes about the OS
The "OS", as it is, is basically just a folder structure where you can arrange pdfs, EPUBs (I haven't tried this), and "notebooks", which are basically collections of pages of your drawing/writing/scribbling.
You can also add things to "favorites" and sort by "type" (notebook/pdf/EPUB).
It's not amazing, especially since it's a bit sluggish to use, but it does the job.
You can get stuff out of and into the reMarkable in a few ways:
- If you create an account in their server and connect it, it automatically syncs to their cloud, and you can get at this data from a desktop app and a mobile app. These are also how you get stuff into it. The syncing here is pretty slow, so I don't use this a lot.
- You can send a mail from the tablet with a collection of pages - either annotated pdf/epub or raw notebook. This is what I do, and it works great!
- There's also a browser extension which turns a website into an EPUB and sends it to the tablet. It works okay, about the same as something like Pocket or Instapaper.
- You can also have the reMarkable convert you handwriting into text and send it as an email. This is actually surprisingly (to me) good, but probably not quite good enough to send a mail without proofreading (you can check and edit the message on the tablet before sending).
The first line in the following picture was converted without error, while the second became "The Quick brown tote jumps over the lazy chef".
I haven't tested this extensively, and I expect it to do worse on other languages than English (you can switch the language used in the setting menu).
Eigil Fjeldgren Rischel
2020
11
10
The physical feel of things
The pen weighs 19g, and feels about like an ordinary ballpoint pen/fineliner in the hand. I would have liked a bit more weight, but it's not a big deal.
The tablet is pretty light. Holding it in one hand is a bit heavy for me - not TOO heavy to use like that, but I probably won't read like that for, say, multiple hours. This is not a big deal though.
The "Book Folio" cover is fine. It's fairly basic, certainly not "worth" 100 EUR - if there was a market for accesories, you could certainly buy a better one much cheaper. It essentially consists of two pieces of sturdy cardboard (I think), covered on the outside by light grey fabric and on the inside by something that feels like felt. The tablet attached to the backside by magnets - these are not that strong, but they do the job. The pen simply attached to the tablet itself by magnets - again, these do the job, but I would have liked them to be a bit stronger. I am slightly worried about the pen falling off in a bag full of mess and getting into trouble. The front piece also attaches by (weaker) magnets when closed, so it doesn't open by itself.
Eigil Fjeldgren Rischel
2020
11
10
Notes about my "workflow"
The basic things I've done with the reMarkable so far breaks down into roughly three categories:
- During video meetings, I've scratched notes on the tablet
- I've put things on the tablet to read, and scribbled in the margins while I did so
- I've used the blank pages (notebooks) to work on.
Eigil Fjeldgren Rischel
2020
11
10
Meeting notes
The device doesn't really shine here. The only advantage it has over writing on the computer is that I can keep the window with people's faces visible while writing, which I could do anyway if I had a second monitor.
There's something nice about paper writing - maybe the flexibility of not being limited to one line after the other? Maybe something physical? That this captures but writing on the computer doesn't, though. But this is a small advantage.
Eigil Fjeldgren Rischel
2020
11
10
Reading and scribbling
This is the real killer app for me. As noted above, you can put your pdfs on the reMarkable and read them. And while reading them, you can scribble on them. I am one of those people who likes to constantly doodle on things, so the eraser really comes in handy here - I can make a scribble, then erase it, so the margins don't get covered in random noise. This is an advantage I hadn't thought of.
After scribbling and highlighting and so on, I usually send annotated pages to my email. Then they can become part of my diginal notes.
I think the fact that this doesn't browse the internet is probably a huge advantage, because it adds a bit of friction to checking twitter, making me distract myself less. Just having a pdf reader that can't check twitter might contribute a huge amount of value.
Eigil Fjeldgren Rischel
2020
11
10
Working on blank paper
As mentioned above, using blank paper to think on is really important to me. On the reMarkable, there's a distinguished "quick pages" notebook with a permanent shortcut to it on the top. This gives you an infinite supply of blank pages to work on.
I also use a few other notebooks to collect thoughts about various projects, but this is not very well-organized.
The biggest drawback here is that the pages are not very large. I think the screen is slightly broader than A6 format (checked by holding up a folded sheet of A4). This is not optimal - I would like to be able to draw bigger things, to keep more thoughts on the page at once. The jury is still out, but I can't imagine the reMarkable will permanently replace paper in this domain for me - at best, it will supplement it.
Eigil Fjeldgren Rischel
2020
11
10
Notes on price & comparison to other products
The reMarkable 2 device itself costs 400 EUR (473 USD at current rates, but prices may differ by region).
This price point is severely misleading, since it's more or less useless unless you buy a pen (you can read on it, of course, but if that's all you want to do I would definitely consider it overpriced).
The basic "Marker" costs 60 EUR, and the "Marker plus" costs 100 EUR.
The only difference is that the plus weighs slightly more and that you can use the rear end to erase.
It's probably best to just consider the base price of the device 460 EUR and ask whether you want to pay 40 EUR for the eraser.
Lastly, you can purchase a cover, or "folio".
Here your options are the basic folio for 80 EUR, which is simply a sleeve that you can slide the reMarkable into,
and the "Book Folio", which is essentially a folder, which magnetically attaches to the back of the reMarkable and wraps around it.
This costs 100 EUR for the basic "polymer weave", and 150 EUR for the black or brown "Premium Leather" version.
These are "overpriced" in the sense that, if there was a market for reMarkable accessories, you would certainly be able to buy much better protective covers for much less. But there isn't, so you're stuck with these, and the question is whether they're worth the money.
You can also buy a reMarkable 1 with basic Folio and (non-eraser) Marker for 349 EUR.
When I bought my reMarkable, I paid a reduced price because I was buying it early. I bought the Marker Pro and the Polymer Weave Book Folio, and ended up paying 460 EUR.
So how much does the reMarkable 2 cost? Based on my experience so far, the built-in eraser is super convenient and well worth the money. You probably want _some_ sort of protective cover, to prevent the screen from getting dings and scratches, but I have to stress that the Folios really seem to be nothing special. If I was buying it again I would probably buy the Book Folio again, just because 20 EUR isn't a lot and it seems a bit incovenient to be taking it in and out of the sleeve all the time.
On that basis, it costs 600 EUR, or ~710 USD.
That's a lot.
For that money you could buy an iPad (329 USD), Apple Pencil ($129), and a smart cover ($50), and still have money left over.
You could use that money to buy extra memory for the iPad, cellular web access (not a killer app for me, but the reMarkable lacks it). You could even buy an iPad Air, if you maybe skimped out on the cover. I didn't look into the Android options, but presumably those are even cheaper. So if you're looking for a "general work tablet", you should probably skip the reMarkable.
If you're looking for the paper feeling, you could consider Paperlike, for just 40 USD (which I have not tried and cannot opine on).
reMarkable really has nothing in the way of tablet features. You can't install any apps (actually, there's a small community of hackers, but nobody is seriously developing third-party software). You wouldn't want to install any of the apps you have on other devices even if you could, because the refresh speed is too low for most "normal" software to really be usable.
You can't browse the internet.
What you can do is basically:
- Write and draw with a stylus like you were drawing on blank paper
- Read pdfs and ebooks, and write on them.
If stuff like that is such an important part of your workflow that getting a significantly better experience than you would doing the same thing on an iPad is worth giving up _all of the iPad's other features_, you should consider the reMarkable. Personally, this is basically the only reason I would really want to own a tablet anyway. In fact, the fact that I can't browse the internet on it is probably a strong feature in its favor, since it makes it harder to distract yourself with twitter.
Another comparison is with the line of products from Boox. The most direct comparison is with the Note Air, which has a similar form factor and price point. There is also Supernote. I have no experience with any of these. It seems that the Boox Note Air is generally more feature-rich than the reMarkable - it actually runs Android, and is thus closer to a "proper" tablet. This review may provide some information, and recommends the Note Air.
I have to say these products are really tempting me, especially the ones with larger screens. If I was buying a new "paper tablet" today, I would seriously consider them.
[^fn:1]: for the record, her opinion of my reMarkable was that it seemed awesome, but it was a nonstarter for her, because it didn't do colors
Eigil Fjeldgren Rischel
2020
11
4
https://erischel.com/eulers-method-is-compositional/
eulers-method-is-compositional
/eulers-method-is-compositional/
Euler's method is compositional
Another day, another post about dynamical systems.
Today, I want to think about _open_ dynamical systems.
You can think of an open dynamical system as a system where
The dynamics are parameterized by some variable (which is supposed to vary with time)
And some function of the state is exposed (maybe to parameterize other systems).
I want to describe two types of open dynamical systems: continuous ones and discrete ones, and show that Euler's method is a compositional mapping between them.
Eigil Fjeldgren Rischel
2020
11
4
Open dynamical systems
A _continuous open dynamical system_ consists of the following data:
- An input set
- An output set
- A state set
- A "dynamics" .
- A "readout" .
This is a continous open dynamical system . Here are natural numbers.
Note that we are constraining our spaces to being . Of course we could have made sense of the above definition for any smooth manifold, asking instead for a map so that for each the map is a section of the tangent bundle. The reason we're looking only at Euclidean spaces is because, to actually use Euler's method,
we need a way of "stepping along the derivative". In our case this is given by simply adding the derivative - in general, we need a map .
A trajectory of this dynamical system is a pair of functions so that .
Given systems , we can compose them to obtain a system .
Suppose the state, dynamics and readout of the two systems are respectively .
Then the state of the composed system is , the readout is simply ,
and the dynamics are .
This corresponds to plugging the output of the first system into the input of the second.
Note that this does not form a category, because there are no identities - there is no way to simply feed the input into the output.
Now for the discrete systems:
A discrete open dynamical system consists of the following data:
- An input set
- An output set
- A state set
- An update function .
- A readout function .
Now the composition is pretty much analogous: given systems ,
the composite has input , output , state space , readout ,
and update .
Again, there are no identities.
Eigil Fjeldgren Rischel
2020
11
4
Euler's method
Recall that _Euler's method_ is a way of solving a differential equation .
Given a starting condition , we fix some step size and move forward one step at a time,
approximating .
This is essentially replacing a continuous dynamic system with a discrete dynamical system.
The state space is the same, and the update function steps forward by the above method.
Formally, if is a continuous dynamical system, define to have
- State space - the same state space, essentially.
- Input space
- Output space
- Update function
- Readout simply given by
Then, we have that is _compositional_ - in other words, if are continuous systems and their composite, then . The proof is essentially just by writing out the definitions.
- The desired identity for in/output, state sets and readout is clear.
- For the update, on the left-hand side the update is given by
.
On the left-hand side, the update is given by - these are of course equal.
Eigil Fjeldgren Rischel
2020
11
4
Higher-order methods
In ODE solving, a "higher-order method" is one that tries to incorporate information about higher derivatives to make a better estimate.
We can view Euler's method as, essentially, calculating by replacing with its first-order Taylor approximation , which we can calculate from the differential equation.
We can often get a better estimate by incorporating higher derivatives - by trying to compute , for example. Essentially, we are incorporating information about how the derivative will change (or how we think it will change) over the interval . If it's going up, then we'll get a higher value than we expected just from looking at the derivative now. If it's going down, we'll get a lower value.
The issue here, of course, is that we can't compute the second derivative directly - the differential equation doesn't tell us what it should be.
That leaves us with two options
- Derive both sides of to get .
Of course, this requires that the function in the equation is actually differentiable, and that you have access to a symbolic representation so that you can compute the derivative explicitly.
This falls in the general class of methods called _multiderivative methods_.
It's easy to see how the above also gives a mapping from continuous systems to discrete ones, although I have not checked the details of compositionality.
- Estimate .
This can then be calculated from the past two steps.
It's not immediately obvious how to make this into a discrete dynamical system. After all, the next state depends not only on the current state, but also on the previous one.
The idea will be to make a state of the discrete system consist of _two_ points in the space - the transition replaces the first with the second, and computes a new second point by the method.
Explicitly, given as above a continuous dynamical system:
- The input and output are as Euler's method
- The state is .
- The readout is (the readout of the "present") state.
- The update is
Note that in this method, we are updating under the assumption of constant input - it may be more appropriate to remember the last input as well, and use that.
In any case, I have not verified compositionality of this method either.
Eigil Fjeldgren Rischel
2020
10
30
https://erischel.com/cofree-dynamical-systems-and-chaos/
cofree-dynamical-systems-and-chaos
/cofree-dynamical-systems-and-chaos/
Cofree dynamical systems and chaos
This blog post largely retraces ideas from Lawvere: Functorial remarks on the general concept of chaos.
I saw this in this tweet from Jade Master, which this blog post is basically an extended version of. Hat tip to her.
Let's try to apply category theory to the study of "dynamical systems".
What is a dynamical system? There are a lot of different versions:
A discrete dynamical system is a set with a map .
A discrete Markov process if a countable set equipped with a stochastic matrix.
A smooth dynamical systsem is a smooth manifold equipped with a section of the tangent bundle .
Today, we'll take the following general view:
Let be a symmetric monoidal category, and let be a commutative monoid in [^fn:1].
Then a -dynamical system is simply an object of equipped with an action of , .
We think of the elements of as "time-shifts", and the composition adds these together.
Using this, we can recover a wide variety of different types of dynamical systems:
Let be the category of sets, equipped with the cartesian monoidal structure. Let .
Then a -dynamical system is exactly a discrete dynamical system in the previous sense.
Let instead be the category of countable sets and Markov kernels, and let be again (with the usual monoidal structure regarded as a deterministic Markov kernel). Then a -dynamical system is exactly a discrete Markov process[^fn:2]
Let be the category of smooth manifolds with the Cartesian monoidal structure, and let .
Then an -dynamical system is _almost_ the same thing as a smooth dynamical system in the above sense.
A -dynamical system picks out a smooth trajectory for each , in a compatible way. This gives a smooth vector field (i.e a smooth dynamical system in the above sense) .
This is not quite a 1-1 correspondence, for example because even smooth dynamical systems in that sense can experience "finite-time-blowup".
For example, if we let , then this a dynamical system is just an ordinary (time-independent) differential equation. If we put , the unique solution given is , which goes to as . So there is no way to find a trajectory, extended for arbitrarily long time, which solves this equation.
But on the other hand, perhaps equations like this are bad and shouldn't be counted. Anyways, they are not dynamical systems in this sense.
Let denote the category of -dynamical systems - their maps are simply -equivariant maps.
There is an obvious forgetful functor .
Suppose is a closed monoidal category. Then the above functor has a right adjoint, which sends to .
acts on simply by "translation", i.e
To be more formal, the map is adjoint to the map
given by multiplying the 2 s, then evaluating the hom.
Proof: We claim that .
Suppose we have a -equivariant map .
-equivariance, of course, means that the diagram
commutes.
The bare map corresponds to a map .
The above commutative square means that this square commutes:
The bottom way around is "let act on , then use the map".
The other one is "multiply the s together, then use the map".
Now let's insert the unit into the right-hand :
Both these squares commute. And the top arrow is just the identity.
In other words the classifying map must equal the other composite, which is "act, and evaluate at ".
Equationally, if , this means .
This means the map is uniquely determined by the map given by evaluation at .
On the other hand, it's not hard to see that any map can be extended to an equivariant map by this method.
This concludes the proof.
Now, what can we do with these "cofree dynamical systems"[^fn:3]?
Here is a cool thing: We can give a categorical definition of _chaos_.
Let be a -dynamical system. Let be a map in on the underlying space of .
We can think of as an "observable": some property of the state which we can measure.
There is a corresponding map of systems which takes each point in to its trajectory of observations
We say is _chaotic_ with respect to if map is an epimorphism.
If we think of an epimorphism as a "surjection", this means every possible sequence of observations is possible. In other words, our current observations don't exclude any possible pattern of future observations.
[^fn:1]: It is not really important that these are symmetric and commutative, but I can't be bothered to keep track of the order, and I don't have any natural non-commutative examples
[^fn:2]: It may have been more natural to consider finite-state Markov processes, but of course isn't finite, so that wouldn't quite have worked
[^fn:3]: Incidentally, there is also a further left adjoint, the "free dynamical system", given by
Eigil Fjeldgren Rischel
2020
10
25
https://erischel.com/chu-spaces-and-linear-logic/
chu-spaces-and-linear-logic
/chu-spaces-and-linear-logic/
Chu spaces and linear logic
A _Chu space over _ consists of a pair of sets , and a function .
A map of chu spaces is a pair of maps so that the diagram
commutes.
This defines a category of Chu spaces, called
You can think of a Chu space as a normal-form game.
is the set of choices available to one player, and is the set of choices available to the other.
The outcome, given the choices and , is .
You can think of a map of Chu spaces as a way of transforming strategies between the two games.
If you would play in the original game, you instead play .
The map in the opposite direction ensures that, no matter what your opponent chooses in the target game, the same outcome could have happened in the domain game - hence your strategy is no worse, in some vague sense (but note that there is no ordering on , so this does not literally make sense).
The cool thing is that Chu spaces are also a model of linear logic, in a way that sort of reflects "Game semantics for linear logic", but with some important differences (which we will see).
Eigil Fjeldgren Rischel
2020
10
25
The duality
The dual of a chu space is simply given by swapping the two sets, i.e .
This defines a self-duality on the category of Chu spaces,
Eigil Fjeldgren Rischel
2020
10
25
The additive connectives
The additive connectives and are simply given by the product and coproduct in the category of Chu spaces.
Let's describe this explicitly:
- Given Chu spaces , their product is given by
, where , and .
This corresponds to a game where the other player chooses one of the games to play, and a choice in that game. You have to choose a strategy for both games, not knowing which will be played.
- The terminal object is , where denotes the empty function.
Interpreting this as a game is slightly weird - it is a game that cannot be played, as the opponent has no moves to make.
(Hence if given a choice between this and another game, the opponent must move in the other game, so that this is a unit for the product, as it should be).
- Given and as above, their coproduct is given by , which is exactly dual to the product.
In other words, here the player chooses a game and a move in it, while the opponent must choose a move in both games.
- The initial object is - the player cannot move in this game.
The additive connectives, in other words, are much as you would expect from the "game" interpretation.
The multiplicative connectives are more weird.
Eigil Fjeldgren Rischel
2020
10
25
The multiplicative connectives.
The tensor of two Chu spaces as above is defined by
,
where .
(Here the last equality is exactly the condition that lies in the pullback).
In other words, to play , the player simply chooses a strategy in each game.
The opponent must choose a strategy in which depends on the player's strategy in - a map ,
and vice versa. These conditional strategies must satisfy the condition that, whether we play using the strategy matching the player's strategy from , or vice versa, we get the same result.
If you squint a bit, this is somewhat like the tensor product of games from before - the opponent can adjust their strategy based on the player's moves in the other game.
But it's also very different.
For example if the two images and are disjoint subsets of , the opponent has no moves in this game.
The tensorial unit is - in this game, the opponent simply chooses the outcome.
Indeed, we have - the player's choice amounts to choosing , while the opponent must choose a map - a point in , and a map , which has to equal the map corresponding to their chosen , so this is no choice at all.
There is a canonical map .
In , the player must choose a strategy and a counter-strategy.
The opponent must choose a map and - of course the canonical choice is the identity.
The outcome of this game is exactly the result of playing the player's two chosen strategies against each other - this is the element that the player's strategy is taken to.
In general, we can identify maps with choices of strategy for the opponent - the point is sent to that strategy, while the map is constrained to being exactly the outcome corresponding to that strategy.
Dually, we can identify maps with strategies for the player.
This of course means that a linear logic proof of the sequent identifies a strategy for the player in the game defined by .
Eigil Fjeldgren Rischel
2020
10
21
https://erischel.com/game-semantics-of-linear-logic/
game-semantics-of-linear-logic
/game-semantics-of-linear-logic/
Game semantics of linear logic
Linear logic is a weird sort of logic.
It's most commonly explained by saying that the "weakening" rule: <object type="image/svg+xml" data="/ox-hugo/20200414104653-blog_posts_570c4c92abfcb8419f3d26233037e0de1c74aecf.svg" class="org-svg">
Sorry, your browser does not support SVG.</object>
and the "contraction" rule <object type="image/svg+xml" data="/ox-hugo/20200414104653-blog_posts_b3da13021d306636a1a258e2de559738f5f41da8.svg" class="org-svg">
Sorry, your browser does not support SVG.</object>.
In other words - you have to use all the assumptions, and you can't use an assumption more than once.
This is usually interpreted in terms of _resources_ - just because I can make a <object type="image/svg+xml" data="/ox-hugo/20200414104653-blog_posts_8582cbf09790ec5906444f9f5df9cd00053cb98e.svg" class="org-svg">
Sorry, your browser does not support SVG.</object> out of two As doesn't mean I can do it with one <object type="image/svg+xml" data="/ox-hugo/20200414104653-blog_posts_a7d92cc965ef976b84b9370a6f429d98bc7d2fb9.svg" class="org-svg">
Sorry, your browser does not support SVG.</object>. And I can't just throw an <object type="image/svg+xml" data="/ox-hugo/20200414104653-blog_posts_a7d92cc965ef976b84b9370a6f429d98bc7d2fb9.svg" class="org-svg">
Sorry, your browser does not support SVG.</object> away that I might not need - I need to find a process for getting rid of it.
However, there's also another way of thinking about linear logics - in terms of _games_.
The combinators of linear logics become ways of combining games into other games.
The interpretation of the deduction rules is now that a proof of the sequent <object type="image/svg+xml" data="/ox-hugo/20200414104653-blog_posts_fbc8e225dbcfabfee571f016d25f90b9b3aca8af.svg" class="org-svg">
Sorry, your browser does not support SVG.</object> should provide a winning strategy for <object type="image/svg+xml" data="/ox-hugo/20200414104653-blog_posts_a7d92cc965ef976b84b9370a6f429d98bc7d2fb9.svg" class="org-svg">
Sorry, your browser does not support SVG.</object>.
Let us be a bit more (but not too) precise.
There are two players. Following convention, we call them the _prover_, P, and the _opponent_, O.
It is understood that we are "on the side" of P, although we could equally well do it the other way - the logic is symmetric enough for that.
A game has a starting player (in many treatments, it is assumed the opponent always goes first, but not here).
Play is usually assumed to proceed back and forth, each player making one move at a time, but this is not important - the most important thing is that in each position, there is a well-defined next player.
Each player has a number of possible moves, only some of which may be legal in a given position - by "position", we simply mean the sequence of moves so far. We generally think of a player as losing when they have to move, but have no legal moves.
The most basic operator is the "dual" or "negation" operator, <object type="image/svg+xml" data="/ox-hugo/20200414104653-blog_posts_bc4947b04e7ea496037197a0809d11da55422409.svg" class="org-svg">
Sorry, your browser does not support SVG.</object>. This simply interchanges the position of the players.
The next operator is <object type="image/svg+xml" data="/ox-hugo/20200414104653-blog_posts_279df3c8f4a8f9a7e4ae8fbdb05c6f1d880f9c99.svg" class="org-svg">
Sorry, your browser does not support SVG.</object>. This game consist of playing <object type="image/svg+xml" data="/ox-hugo/20200414104653-blog_posts_a7d92cc965ef976b84b9370a6f429d98bc7d2fb9.svg" class="org-svg">
Sorry, your browser does not support SVG.</object> and <object type="image/svg+xml" data="/ox-hugo/20200414104653-blog_posts_8582cbf09790ec5906444f9f5df9cd00053cb98e.svg" class="org-svg">
Sorry, your browser does not support SVG.</object> simultaneously - the player must move in whatever game the opponent just moved in, while the opponent can switch games at will.
Clearly, a winning strategy for this game, for the player, entails a winning strategy for each component game.
In this sense, this game is like logical "and".
But there's also another operator like "and", which is written <object type="image/svg+xml" data="/ox-hugo/20200414104653-blog_posts_cd339c6a9fd60f255d5641e7b67849d12862ec66.svg" class="org-svg">
Sorry, your browser does not support SVG.</object>.
This is played as follows: the opponent chooses a game, then that game is played to completion, and its winner wins the whole game.
Here is a difference between these operators: There is a canonical strategy for <object type="image/svg+xml" data="/ox-hugo/20200414104653-blog_posts_c224d906e554f6971f4d8beea3fbf24f53936870.svg" class="org-svg">
Sorry, your browser does not support SVG.</object>, but not for <object type="image/svg+xml" data="/ox-hugo/20200414104653-blog_posts_c7ea9bf60399dae5f34e826d4f1a8ecc0a4a6030.svg" class="org-svg">
Sorry, your browser does not support SVG.</object>.
How does this work? To play <object type="image/svg+xml" data="/ox-hugo/20200414104653-blog_posts_c224d906e554f6971f4d8beea3fbf24f53936870.svg" class="org-svg">
Sorry, your browser does not support SVG.</object>, we must play the opponent's side in <object type="image/svg+xml" data="/ox-hugo/20200414104653-blog_posts_d3860e22107b32eb22a9449fa3807d8f08cea676.svg" class="org-svg">
Sorry, your browser does not support SVG.</object>.
Assume wlog that the player goes first in <object type="image/svg+xml" data="/ox-hugo/20200414104653-blog_posts_a7d92cc965ef976b84b9370a6f429d98bc7d2fb9.svg" class="org-svg">
Sorry, your browser does not support SVG.</object>.
We have the player start playing in <object type="image/svg+xml" data="/ox-hugo/20200414104653-blog_posts_a7d92cc965ef976b84b9370a6f429d98bc7d2fb9.svg" class="org-svg">
Sorry, your browser does not support SVG.</object>, then copy this move into <object type="image/svg+xml" data="/ox-hugo/20200414104653-blog_posts_bc4947b04e7ea496037197a0809d11da55422409.svg" class="org-svg">
Sorry, your browser does not support SVG.</object>.
Whatever they play in response, we go to <object type="image/svg+xml" data="/ox-hugo/20200414104653-blog_posts_a7d92cc965ef976b84b9370a6f429d98bc7d2fb9.svg" class="org-svg">
Sorry, your browser does not support SVG.</object> and play that as our response to them, and so on.
We're guaranteed to win (exactly) one of the games, since the two games will be exactly mirrored.
If this doesn't make sense, imagine playing two chess games against another person - one as black, one as white.
Starting with the game in which they're white, they make a move. You copy this move on the other chessboard.
They respond with black, which you also copy on the fist board.
When they mate you at some point on one board, you copy this immediately, winning the other game.
This strategy won't work for <object type="image/svg+xml" data="/ox-hugo/20200414104653-blog_posts_c7ea9bf60399dae5f34e826d4f1a8ecc0a4a6030.svg" class="org-svg">
Sorry, your browser does not support SVG.</object>.
Here we would have to decide at the beginning which game we have a winning strategy in, then choose that one.
In logic, this means that the sequent <object type="image/svg+xml" data="/ox-hugo/20200414104653-blog_posts_8964387a593f0de3a11f74446080e77066b64d2e.svg" class="org-svg">
Sorry, your browser does not support SVG.</object> is provable, but <object type="image/svg+xml" data="/ox-hugo/20200414104653-blog_posts_1e6005344886e5204b0108a5d3703657626bc38c.svg" class="org-svg">
Sorry, your browser does not support SVG.</object> isn't.
So far, we've only looked at the two forms of conjunction.
There is also two forms of disjunction, given simply by <object type="image/svg+xml" data="/ox-hugo/20200414104653-blog_posts_332bfcca70f28fa5e5f48759a18df39665e5212c.svg" class="org-svg">
Sorry, your browser does not support SVG.</object>, <object type="image/svg+xml" data="/ox-hugo/20200414104653-blog_posts_b6e991d197297486fb377f8b2f7a71d7df9a0212.svg" class="org-svg">
Sorry, your browser does not support SVG.</object>.
We could restate the above discussion as "<object type="image/svg+xml" data="/ox-hugo/20200414104653-blog_posts_67c4e3051ed267d97a37f2525aea5d5118ea2b30.svg" class="org-svg">
Sorry, your browser does not support SVG.</object> is not provable, but $is".
This means we have a form of excluded middle, but another form of excluded middle doesn't hold.
Now what is the meaning of a general sequent of the form <object type="image/svg+xml" data="/ox-hugo/20200414104653-blog_posts_b995dd0c8fb474bb2182224fbeab583cfacd8a49.svg" class="org-svg">
Sorry, your browser does not support SVG.</object>, where both <object type="image/svg+xml" data="/ox-hugo/20200414104653-blog_posts_82b14cc542ef22dba87993b12dd6d6d310f947d7.svg" class="org-svg">
Sorry, your browser does not support SVG.</object> and <object type="image/svg+xml" data="/ox-hugo/20200414104653-blog_posts_a33a2d6e25b53b1938ccfc129cc9544b9628375b.svg" class="org-svg">
Sorry, your browser does not support SVG.</object> are multisets.
We can interpret such a sequent as asserting the existence of a strategy for the game <object type="image/svg+xml" data="/ox-hugo/20200414104653-blog_posts_3415bd826597d728f3ee0489dbfa6c59ba7f9f84.svg" class="org-svg">
Sorry, your browser does not support SVG.</object>.
A proof is supposed to identify a specific strategy for the game.
This also means that any sequent can be replaced by a one-sided one, which means linear logic can be expressed all in terms of one-sided sequents (which is indeed true).
Now we can think about the lack of contraction - we can examine simply the fact that <object type="image/svg+xml" data="/ox-hugo/20200414104653-blog_posts_4e379c847bf45683840230c2b93ef93f20c24be8.svg" class="org-svg">
Sorry, your browser does not support SVG.</object> is not a provable sequent.
To prove this sequent, we would want a strategy for <object type="image/svg+xml" data="/ox-hugo/20200414104653-blog_posts_404495e2a205415c8b6fc83ff9a40bce2b9ef3c4.svg" class="org-svg">
Sorry, your browser does not support SVG.</object>.
We can think of this again in the case where <object type="image/svg+xml" data="/ox-hugo/20200414104653-blog_posts_a7d92cc965ef976b84b9370a6f429d98bc7d2fb9.svg" class="org-svg">
Sorry, your browser does not support SVG.</object> is chess.
Imagine that instead of two games, there are three, two where you are black.
You can only respond on whatever black board where your opponent just moved as white, or move as white on the board where you are white (forcing your opponent to respond there). And when you respond as black, your opponent can switch to the other black board.
You can only mirror one of these onto the game where you're white, so the strategy-stealing trick no longer works.
On the other hand, weakening _is_ valid in this semantics. We do have a strategy for the game <object type="image/svg+xml" data="/ox-hugo/20200414104653-blog_posts_afddd41d83bccf2bec783f5e313950c75d967c4b.svg" class="org-svg">
Sorry, your browser does not support SVG.</object>.
Here you have two white boards and one black, and you can simply ignore the white board you don't need, since you now have full control over which board you move on.
This is all written out in Blass: A game semantics for linear logic.
Eigil Fjeldgren Rischel
2020
10
14
https://erischel.com/universal-properties-and-compositionality/
universal-properties-and-compositionality
/universal-properties-and-compositionality/
Universal properties and Compositionality
Continuing the train of thought from this tweet, I compare and contrast two perspectives on the philosophy of category theory: that it's about describing how things can be composed of other things ("compositionality"), and that it's about describing things in terms of their transformations into other things ("universal properties").
Eigil Fjeldgren Rischel
2020
10
14
Some uses of category theory
Category theory is an amazingly successful tool for pure mathematics.
Since its introduction by Eilenberg and MacLane in Generalized Theory of Natural Equivalences, it has completely transformed algebraic topology and algebraic geometry.
Let's study some examples of this:
Eigil Fjeldgren Rischel
2020
10
14
The Brouwer Fixpoint Theorem
This well-known theorem says that any continuous map [^fn:1] must have a fixpoint, i.e a point so that .
The usual way to prove this is to suppose for contradiction that we have a map with no fixpoints.
Then by drawing a line from to and seeing where it intersects the boundary of , we find a continuous map with the property that when .
Now we have to prove that there is no such continuous map.
Luckily, there happens to exist a _functor_, , with the properties that
.
If there really was a as above, so that the composition was the identity, then by applying to it we would find a map so that was the identity - which is clearly impossible.
If we wanted to summarize this argument, we could say
- is not a retract of (algebra)
- _Functors preserve retracts_ (category theory)
- Therefore is not a retract of . (topology)
Eigil Fjeldgren Rischel
2020
10
14
Idempotent monads, aka localizations
It's very common in algebraic topology that you have some class of morphisms in your category that you want to treat as isomorphisms.
First of all, category theory tells us that there is essentially a unique way to make sense of this:
The desired category should have the following universal property: given a functor which sends every to an isomorphism, there should be a unique extension over , as in this diagram:
This formalizes the intuition that is , with the maps turned isomorphisms, and nothing else changed.
Category theory (the Yoneda lemma), applied to the category of categories, tells us that this determines up to equivalence of categories. Under quite weak assumptions, this category can be constructed.
But understanding the category is usually very difficult.
However, in good cases, something miraculous happens, and appears as a subcategory of itself!
As a very simple example, we can consider the category of abelian groups.
Call a homomorphism a "rational equivalence" if
- If , then for some
- For each in the image, there exists so that .
Call the category of rational equivalences . Then is equivalent to the subcategory consisting of -modules - those abelian groups where each multiplication by map is a bijection.
The way to construct this in general is to consider the subcategory of _-local objects_ - those objects where, for every in , the map is a bijection.
If every object admits a map with and -local, then the construction assembles into a functor, which exhibits the subcategory of -local objects as .
In this case, part of the utility of category theory is it lets us ask this question in the first place.
But beyond that, it gives us access to the idea of, to put it technically, asking for objects that represent functors with certain properties (for example, those that invert certain morphisms) - in general terms, to describe objects in terms of their relations to other objects.
(The really powerful uses of this technique involve doing something more complicated to model categories, or, equivalently, applying it to -categories. But I won't get into that here - see e.g. my undergraduate thesis or section 5.2.7 of Higher Topos Theory.)
Eigil Fjeldgren Rischel
2020
10
14
ZX Calculus
ZX calculus is a graphical language for quantum computing, introduced by Coecke and Duncan in a 2008 paper.
It is essentially a flavor of string diagrams with special operators and rewriting rules to describe the operations in quantum computing. A diagram in the ZX calculus can be interpreted as a linear map between Hilbert spaces.
I won't delve too much into the ZX calculus, since I'm not familiar with it beyond a surface level, but I will note that it's probably one of the more successful applications of category theory so far - I count 88 publications listed on <https://zxcalculus.com>.
Eigil Fjeldgren Rischel
2020
10
14
Open Reaction Networks
Open Reaction Networks, described there by John Baez and Blake Pollard, are a compositional version of reaction networks
Reaction networks describe various notions of "transition system", where you have a number of different "things" or "species" and various ways those things can be transformed into each other, "transitions".
An open reaction networks is augmented with the designation of certain places as inputs and outputs.
The composition operation identifies the outputs of one with the inputs f another, gluing them together in a single reaction network.
Baez and Pollard construct a compositional semantics for reaction networks, which takes a reaction network to an "open dynamical system" describing the dynamics of the outputs, in terms of a function specifying the inputs over time.
Eigil Fjeldgren Rischel
2020
10
14
Compositionality and universal properties
The phrase "universal property"[^fn:2] appears neither in the original ZX calculus paper, nor in Graphical Structures for Design and Verification of Quantum Error Correction, which is one of the more successful applications (both of the ZX calculus and of category theory in general, outside pure math). Clearly they are not that interested in characterizing objects by their mapping properties - indeed, since the objects in this case are just finite-dimensional Hilbert spaces, hence direct sums of , characterizing such a thing is not that interesting.
The point is that in the classical domains of category theory, the primary thing is the objects, and how they map to one another. When we think about maps, we are usually just looking for an abstract machine that will guarantee the existence of a map with such-and-such properties, not in looking "inside" the map, so to speak, to see what it does.
This in contrast with the ZX calculus, where the chief point is that we want a method for writing down maps between Hilbert spaces and understanding how they work.
So also open Petri nets - the interesting structure here is (obviously) the _Petri net_, not the finite sets which serve as objects.
In this domain, the chief interest is in the _functors_ - we want to know, for example, that a certain "black-boxing" construction from the category of Petri nets to the category of (say) linear relations preserves the composition.
Eigil Fjeldgren Rischel
2020
10
14
Synthesis: Double categories and operads
As remarked by Jules Hedges here, the "compositionality" above is really just about operation-preserving maps between algebras of some operad - the fact that the operad (or a suboperad) is the operad for categories[^fn:3] is not really important.
So we can consider the operad where
- The types (or colors) are pairs of finite sets
- The operations are generated by a composition operation , an identity operation , and optionally a "tensor" operation and some more operations for the unitors and associators in the monoidal structure.
- Quotiented by the axioms of a monoidal category.
Then the _set_ of open petri nets is an algebra for this operad, and we're essentially looking for an algebra homomorphism into a different algebra.
This perspective has a number of advantages:
- When mapping into a situation where a slightly different structure exist, we can describe this by adding an operad homomorphism to the mix.
- We can take algebras in _categories_ to get categories where the _objects_ are the systems we're interested in.
This can give us some interesting universal properties - for example, the initial Petri net (with given input-output sets ) would seem to be the net with exactly as places, and no transitions.
We can go even further and consider morphisms between systems with different interfaces.
Presently the best framework for this seems to be double categories, which means we're going back to the operad for categories[^fn:4], but that's fine.
For example, in Morphisms of Open Games, Jules Hedges constructs a double category of open games, and describes limits and colimits in the "vertical category of morphisms" - i.e. in the category where objects are open games and maps are morphisms _between_ them.
In fact, various technologies for describing "open systems", like the Structured Cospans of Baez and Courser, already output a double category. And in fact the composition in these systems already involves some sort of universal property, being described as a pushout.
Eigil Fjeldgren Rischel
2020
10
14
Synthesis: Syntactical categories, Lambda Calculus
Above we have thought about many different situations where we're interested in studying the "behaviour", in some sense, of complex systems, and we try to understand this behaviour as a functor in a category of such systems.
Perhaps the most famous example of this is the notion of _functorial semantics_ from logic (indeed the term "functorial semantics" seems to describe the black-boxing functor quite well).
The general pattern is that we have a _syntactical category_ where the morphisms are terms in some syntax, subject to some equivalence relation (often generated by some set of syntactical rewriting rules), and this is mapped into some sort of "semantics" category.
Here it very often happens that the necessary structure to interpret the syntax corresponds to some more or less natural categorical structure, and the syntactical category is the initial such category (so that there is a well-defined interpretation of the syntax in each suitable category).
An example of this is simply typed lambda calculus, or STLC.
I guess it's well-known that the simply typed lambda-calculus can be interpreted in any cartesian closed category.
Each type is mapped to an object , with .
Then a judgment , where ,
is mapped to a morphism , by inducting on the proof of the judgment using the projections
and the hom-product adjunction.
One can show that the _initial_ Cartesian closed category with a certain set of objects and points - we might say the CCC _generated_ by that data - can be described as having objects the contexts, and morphisms the judgments, in STLC with those constants.
But this is somewhat interesting - the syntax of lambda calculus is "about writing down functions", just as the ZX calculus.
Yet there are also universal properties in the mix - the fact that the functor has a right adjoint for all .
Across many areas of categorical logic, similar things happen.
[^fn:1]: It actually holds for for all , but let's keep things simple
[^fn:2]: A lot of things are described as _universal_, but this refers to the important property that any quantum computation can be described in ZX-calculus
[^fn:3]: There isn't really an operad for categories - what I describe here is for "categories with a specific set of objects" - but that's not so important
[^fn:4]: There are ways to generalize this - David Jaz Myers has studied monoidal double functors into a double category of categories, which seems to be a version of this (in some sense, this uses the fact that any operad can be upgraded to a symmetric monoidal category). See eg this talk.
Eigil Fjeldgren Rischel
2020
10
13
https://erischel.com/left-adjoints-preserve-colimits/
left-adjoints-preserve-colimits
/left-adjoints-preserve-colimits/
Left adjoints preserve colimits.
Let's prove a classical theorem (Emily Riehl's favorite!) from category theory: Right adjoint functors preserve limits.
So let's assume we have categories , functors , and a natural bijection
.
Let's also fix a diagram from some index category .
Now recall that a limit of is an object , equipped with maps , so that every triangle
commutes, and so that, given another set of data with the same property, there is a unique map so that .
(I am sort of assuming you already know about limits, and just writing the definition above for convenience).
The proof idea, very weakly formulated, is that the adjunction lets us control maps into , and the universal property of the limit is precisely about maps into .
Let's first make completely precise the claim: it is that, supposing forms a limit of , then also forms a limit of . Note that this makes sense - really is a map from to .
Now let's fix another cone on .
We are attempting to construct a map . This is equivalent to a map .
We're going to construct _that_ map by constructing maps .
Where do we get those maps? We apply the adjunction again - they are simply the mates of the maps we started with.
Now we need to confirm that the map actually does make the desired triangles commute.
This follows from naturality of the adjunction, using the same property for :
Consider the composite .
By naturality of the adjunction (with regard to postcomposition), the mate of this map is the composite .
By construction, this equals the input map , which is the mate of the original .
Hence the composite has the same mate as , so equals it.
The second thing we need to prove is that this map is unique with this property.
We can do this by noting that, given two distinct map with this property, their mates are distinct (because the correspondence is bijective), and both have this property (by the same argument used above).
This establishes the claim.
Eigil Fjeldgren Rischel
2020
10
11
https://erischel.com/the-homotopy-theory-of-groups/
the-homotopy-theory-of-groups
/the-homotopy-theory-of-groups/
The homotopy theory of groups
Context: Krause and Nikolaus: Group Theory for Homotopy Theorists (pdf).
Krause and Nikolaus develop group theory using model categories (well, one model category).
This is obviously a joke, but I think it _is_ a very useful pedagogical joke. So I'm going to go through it and try to explain what's happening.
Eigil Fjeldgren Rischel
2020
10
11
Group presentations
If you've taken a course on group theory, you've probably learned about _presentations_ of a group.
Here are some examples:
-
-
- .
On the left of the pipe, we have a set of _generators_.
On the right of the pipe, we have a different set of _relations_ - these are "group words" in the generators, i.e words involving both the generators and their inverses.
The meaning of such a "presentation" is that it describes a group, namely the quotient of the free group on by the relations.
Every group has a presentation (in fact, many), for example given by taking and given by all words which evaluate to in . This leads to the idea that one could _define_ group theory out of presentations. This seems to work out well - in fact, given groups, if we take their "canonical" presentations as above, a group homomorphism is exactly the same thing as a function which takes those words in which evaluate to to words in which evaluate to .
However, this doesn't work in general - most group presentations don't have "enough generators" to represent all group homomorphisms into the presented group. And there are many maps which "should" be group isomorphisms which don't have an inverse on the level of presentations.
Eigil Fjeldgren Rischel
2020
10
11
Model categories
The basic things you can do in a model category are:
- Invert certain maps that "should" be isomorphisms (or "equivalences")
- Replace an object with a "better behaved" one which is "equivalent" in that sense.
Let's take a more classical example from "real" homotopy theory: Simplicial sets.
In this context, we think of a simplicial set as a "model" for a topological space, its geometric realization .
A map is a simplicial homotopy equivalence if the map of spaces is a homotopy equivalence of spaces.
However, in many cases, such an equivalence can't be inverted, even up to (simplicial) homotopy.
Similarly, there are often maps between geometric realizations which can't be represented by maps between the simplicial sets, even up to homotopy.
The basic solution to this is to work with certain nice simplicial sets called _Kan complexes_. A Kan complex has "enough simplexes" so that all the maps between them that "should" exist, do.
Now we _could_ simply work with the category of Kan complexes - but this is an inconvenient category. Its main deficiency is probably that it does not have all colimits. This is compared to the full category of simplicial sets, which is as nice as they come.
So we use a powerful piece of technology called a _model structure_, making simplicial sets into a model category.
(This is usually called the Kan model structure, or the Kan-Quillen model structure).
The upshot of this is that every simplicial set is homotopy equivalent to a Kan complex, in a very structured way which lets you use the technically convenient structure of the whole category of simplicial sets to describe "the homotopy theory of Kan complexes".
Eigil Fjeldgren Rischel
2020
10
11
Back to groups
Now we are going to try to adapt the above to groups.
The way to do this is to describe the _trivial cofibrations_ - essentially, these will be those maps which are "morally group isomorphisms".
Using the machinery of model categories, we can get away with specifying only a small "generating family".
These are on page 2 of the paper.
Now here's a fun fact: this small list pins down exactly the "proper" presentations of groups, in this sense:
**Theorem**: The following two are equivalent for a presentation
- The map is bijective, and contains every word which is zero in .
- For every diagram
if is in the generating list, there exists an extension like the dashed arrow making the triangle commute.
**Proof**:
First, it's easy to check for each type of generating arrow that this holds:
- and must be sent to the same element (by injectivity), so we can simply send there as well.
- We can extend the map by sending to whatever element evaluates to.
- The remaining three maps simply add relations which are derivable from the existing ones - hence their images must already be in .
Now for the other direction, suppose go to the same element in .
Then the map does not have an extension over the first generating map.
Hence that map is injective.
Suppose is not in .
Then the map does not extend over - there is nowhere to send (since it must be sent to something that goes to , for the property to hold).
The statement that contains every word that goes to zero means that is stable under certain operations, which can similarly be proved from the morphisms.
It follows from the general theory of model categories that there always exists a map into a "nice" (in technical terms, fibrant) presentation, which is itself a trivial cofibration.
Understanding what the trivial cofibrations are in technical terms is more complicated - an argument going in the other direction from the one above will show that they are exactly those which correspond to group isomorphisms, but this might be a bit tricky.
Eigil Fjeldgren Rischel
2020
10
9
https://erischel.com/category-of-computable-categories-with-runtime/
category-of-computable-categories-with-runtime
/category-of-computable-categories-with-runtime/
A category of computable functions with runtime
See: Giorgios Bakirtzis and Christian Williams: Turing Categories.
Turing Categories describe _computability_.
I want to find a category to work with _complexity_ instead.
This is a stab at it.
Fix a universal Turing machine and an encoding of the natural numbers.
Of course, this lets us speak of _computable_ functions (and these don't depend on the choice of Turing machine).
But fixing a specific machine also lets us speak of the _runtime_, in steps, of a program with input .
Write if eventually terminates with output , and write if it takes steps.
Now we want to define a category as follows:
The objects are subsets of the natural numbers.
The maps are equivalence classes of programs under an "asymptotic equivalence" relation.
The composition is "the obvious thing".
What I mean by asymptotic equivalence is this: the two programs halt on the same inputs, produce the same outputs, and run in the same amount of time up to a constant factor.
To be more precise, if there exists a constant so that, when and ,
and .
Then if and .
This corresponds to our natural definition of "same behaviour, and same -class".
Now, we want to construct the composition simply to have and .
(This means that if halts with output , and halts with output , then halts with output , and has the runtime described above, otherwise it doesn't halt).
It's not completely trivial that this is possible - essentially, the "overhead of composition" must be smaller than the runtime of the larger of and .
We must add this as a requirement to our Turing machine - luckily, this seems to be satisfied by most sane models of computation.
Call the category so defined .
Eigil Fjeldgren Rischel
2020
10
9
Comp as a restriction category
Given a program , we can define as follows: to compute , run until completion, then return no matter what.
Certainly any universal Turing machine admits this construction. Does this define a restriction category?
Recall the axioms of a restriction category:
1.
2.
3.
4.
(If these compositions are well-defined).
1. Clearly has the same values as .
It runs in almost exactly twice the runtime of , plus whatever the overhead of composition and the construction is.
It is not actually completely obvious that this is small enough for _every_ universal Turing machine.
Nevertheless, the assumption that for some constant seems rather harmless, which suffices here.
2. and , again, clearly have the same values. With the assumption from above, the runtime is also asymptotically the same.
3. Once again it's easy to see that the two maps have the same values.
The first one runs in , and so does the second one, hence they have the same (asymptotically) runtime.
4. Here the claim about equivalent runtimes is a bit more subtle.
The first one takes time (up to asymptotic equivalence).
The second one takes .
But these two are equivalent - the second is at most twice the first.
Hence is a restriction category.
Eigil Fjeldgren Rischel
2020
10
9
Comp as a Cartesian restriction category.
Do "restriction products" exist in .
This would seem to rely on a _constant-time_ bijection .
I am fairly sure this doesn't exist - since the encoding size of a natural number grows as the logarithm, it would seem that such a bijection must have unbounded runtime.
This may be remedied by passing to a less restrictive form of equivalence.
For example, suppose we can find a logarithmic-time implementation of the above bijection.
Then we can alter the definition of so that as long as there exists so that (and they have the same values).
There are other reasons to prefer this equivalence relation.
It is not in general true that universal Turing machines can simulate each other with _constant_ overhead.
This means that the choice of UTM made at the beginning of this post is actually significant.
But it _is_ true that they can simulate each other with _logarithmic_ overhead. [^fn:1]
This means the choice of machine is unimportant, making for a much nicer theory.
The construction of this correspondence does seem to be the only obstruction to equipping with a Cartesian restriction structure, in the "obvious" way (where is the subset of which corresponds under this bijection to a pair .)
Eigil Fjeldgren Rischel
2020
10
9
Comp as a traced category
Can we define a trace operation on ?
This would be an operation .
The idea would be that to evaluate on , you run to obtain , then run to obtain and you keep doing this until you find a fixpoint where then the resulting is your output.
(If this never terminates, you simply don't halt on that input).
The obvious problem is that the choice of is non-canonical.
We might want to let only be defined when this input doesn't matter - but this property is not obviously computable! (We would have to check every possible ).
This probably precludes from satisfying the axioms of a trace. I'm not sure, but I think the issue will appear in the axiom .
The right-hand side corresponds to applying ,
then , then proceeding like that until we find a fixpoint .
In the second, we first define a function by applying to find , then going until we find a fixpoint .
Then we apply _the function to defined_, call it , with to find , and so on.
In other words, we first find a fixpoint for , and use that fixpoint to produce .
Then we find a fixpoint for and so produce , and so on.
This does eventually produce a pair which form a fixpoint for , but not necessarily the same one as the "simultaneous" procedure.
This problem is exactly fixed if we only allow those s where the starting input doesn't matter - then there is a unique fixpoint, and we are happy.
The second problem, of course, is that the two procedures defined above have _very different runtimes_.
This problem does seem to be baked into the axioms of a traced monoidal category.
This is perhaps not surprising - these axioms are not saying that both sides are equally fast ways of computing the result, just that the results are equal.
So perhaps we should look for some alternative "complexity-sensitive" version of these axioms that make sense in our context.
On the other hand, the temptation to regard a string diagram with loops as describing a meaningful program is very strong. This would suggest that we need a different version of , or perhaps a different traced structure, that makes the two sides of the equation equal.
Eigil Fjeldgren Rischel
2020
10
9
Comp as a Turing category
Comp is, first of all, not a Turing category, because undecidable subsets of the natural numbers won't be retracts of them.
If we fiddle with the definition to fix this issue (we can remove all other objects than and , for example), it might actually be a Turing category - because of the fact that universal Turing machines can simulate each other with logarithmic overhead.
Eigil Fjeldgren Rischel
2020
10
9
Further questions
- What is the general definition of which is an example?
- Does this actually help at all?
- Is there a (family of) model(s) of computation which allows us to A: make a category with useful internal structure, B: without fiddling with various notions of asymptotic performance.
[^fn:1]: See eg Wikipedia
Eigil Fjeldgren Rischel
2020
10
4
https://erischel.com/localizations-categories-dynamical-systems/
localizations-categories-dynamical-systems
/localizations-categories-dynamical-systems/
Localizations of categories of dynamical systems
See also: Jade Master: Dynamical Systems With Category Theory? Yes!, This tweet by me.
Eigil Fjeldgren Rischel
2020
10
4
Discrete dynamical systems
A discrete dynamical system consists of a set and a time-step function .
It's clear that this is exactly the same thing as a -set, i.e a set with an action of the monoid [^fn:1].
A "morphism of discrete dynamical systems" is just the obvious thing, namely a map which preserves the action - an equivariant map.
For now, we will restrict ourselves to those dynamical systems with "time-reversible dynamics" - that is, those where the action is a bijection.
The subcategory of such things inside is equivalent to - a dynamical system is time-reversible if and only if the action of extends to an action of .
Now, we will be interested in _localizations_ of the category .
One important family of such localizations comes from the group homomorphisms .
This homomorphism gives a functor .
In fact, this functor is fully faithful, and its image is exactly the subcategory of -periodic dynamical systems - i.e those for which for all .
Moreover, this functor admits a left adjoint .
It takes a dynamical system to , where is the equivalence relation generated by and is the induced map.
The order structure of this family of localizations is exactly the division order.
By which I simply mean, if , then an -periodic dynamical system is also -periodic, and if every -periodic dynamical system is -periodic, then (proof of the second implication: consider the dynamical systen ).
Using these, we can form a sort of "-adic completion" of any dynamical system, as the limit
. The functor is a localization.
The -adic completion of the integers with translation action is exactly the -adic integers (with translation action).
Since each system acquires a canonical action of , it would seem that probably acquires an action of the -adics .
However, to make sense of this, it's probably best to work in a category of topological spaces with continuous group actions.
I would guess that the actions assemble to a unique _continuous_ action, although I have not checked it.
The family of localizations is jointly conservative on _finite_ dynamical systems (since each orbit is an -period for some ). However this fails in a predictable way in the infinite case, where we can't distinguish between and . To remedy this, one would need some sort of "rationalization" of dynamical systems.
However, there we run into the issue that the functor is not fully faithful - a set has a chosen "half action", the action of on , but this action is not necessarily uniquely determined by the action of .
Eigil Fjeldgren Rischel
2020
10
4
Smooth dynamical systems
We take a somewhat unorthodox approach and let a _smooth dynamical system_ be a smooth manifold with an action of the Lie group . (Again, we are looking at time-reversible systems).
Given a discrete subgroup of , the quotient is again a Lie group,
and we obtain a fully faithful inclusion .
The discrete subgroups of all have the form for some , and the quotient is always diffeomorphic to .
As above, passing to this quotient corresponds to considering -periodic systems.
In this situation, there no longer exists a left adjoint, for annoying reasons.
The universal property of the left adjoint, if written out, tells us it should take a manifold to the quotient of the action by .
Consider the normal additive action of on .
If we take to be an irrational number, the orbits of the action are dense in ,
and the quotient is not even a topological manifold.
One would hope this problem can be solved by something like "derived manifolds", but I haven't looked into that yet.
[^fn:1]: My natural numbers include 0
Eigil Fjeldgren Rischel
2020
10
2
https://erischel.com/cheap-nonstandard-analysis/
cheap-nonstandard-analysis
/cheap-nonstandard-analysis/
Cheap nonstandard analysis
Terry Tao: A cheap version of nonstandard analysis.
MathOverflow: Does Cheap Nonstandard analysis take place in a topos?
(Answer: Yes, but an elementary topos, not a Grothendieck topos).
This is partially a summary of Tao's blog post, partially a small discussion of way LEM fails for cheap nonstandard reals.
Eigil Fjeldgren Rischel
2020
10
2
What is "nonstandard analysis"?
In "normal" nonstandard analysis, we consruct the "nonstandard reals" as an ultrapowwer of the ordinary reals with regards to some nonprincipial ultrafilter .
This means that a nonstandard real is a sequence of reals, quotiented by the equivalence relation which identifies two sequences if the set of naturals where they agree is in the ultrafilter - in symbols,
This set inherits all the structure of the reals - where addition and multiplication is pointwise, and if . In fact, it is even a totally ordered field, since first-order properties pass to ultrapowers by Łoś's Theorem.
It also contains a copy of the real numbers, where is identified with a constant sequence. We call these the _standard reals_.
( is not complete, since this is a second-order property.)
contains "infinitesimals", like the sequence - this is a positive nonstandard real number, but less than every positive standard real number. Call this number .
It also contains "infinite numbers", like the sequence - this is a nonstandard real wich is larger than every standard real.
Given a function , we can ask for the value .
This is a well-defined nonstandard real, since is not zero.
In fact, it is the nonstandard real .
If comes from a function which is differentiable, this sequence converges to .
This means that the difference between the nonstandard derivative and the normal derivative is an infinitesimal, in the sense that it is smaller than every positive and larger than every negative standard real.
One can show that is the only standard real with this property - so this gives a definition of the derivative (which does not depend on the choice of ).
Nonstandard analysis takes this idea and runs with it to develop various parts of analysis using these infinitesimals.
Eigil Fjeldgren Rischel
2020
10
2
Cheap nonstandard analysis
There are a few problems with this:
- The existence of a non-principal ultrafilter on requires some "unfortunate" axiom like the axiom of choice or somesuch. (The existence of non-principal ultrafilters on all infinite sets is called the ultrafilter theorem, and is strictly weaker than the axiom of choice, but still considered a bit suspicious - for instance it implies the existence of nonmeasurable sets).
- According to Tao, it's sometimes difficult to extract a quantitative bound from an asymptotic theorem proved with nonstandard analysis - he links this to the first point.
However, these difficulties disappear if we instead use the Frechet filter, consisting of all cofinite sets, on .
In other words, a cheap nonstandard real is a sequence of reals, quotiented by the equivalence relation that if for sufficiently big.
Note that the Frechet filter is not an ultrafilter!
Hence Łós' theorem does not hold, so the cheap nonstandard reals fail to inherit all the good properties of the ordinary reals.
The easiest example of this is that they are not a field.
To see this, consider the cheap nonstandard real .
This is not equal to zero, because there are infinitely many ones.
On the other hand, it is also not invertible: if we multiply it with the cheap nonstandard real , we get . This is not equal to one, because there are infinitely many zeroes.
The way this would work for an ultrafilter is that either the set of even numbers or odd numbers would be in the ultrafilter. So either is zero or one - in either case, it's clearly invertible.
It turns out the parts of Łós' theorem that specifically fails for non-ultra filters are about statements - disjunction, and - negation. Hence the statement doesn't hold for the quotiented product, even though it holds for the ultraproduct.
However, this shouldn't concern us _too much_, because we can basically view this as a faily of LEM (in a way that I will explain), and LEM is another one of those slightly suspicious axioms.
In what sense does LEM fail for cheap nonstandard reals?
As mentioned, the nonstandard reals are a model of the language of ordered fields.
The way this works is that any formula in that language can be interpreted for a nonstandard real by asking if holds for a set in the ultrafilter - for short, whether .
A priori, this gives two ways of intepreting a formula like .
We can ask whether , or we can ask whether .
The second is the "correct" semantics for first-order logic, but the first is usually more convenient. Luckily, for ultrafilters they are the same.
We can try to use this approach with a non-ultrafilter, like the Frechet filter . But now there's a difference! If we let , it's true that , but not true that .
A similar problem crops up for negation.
In this way, LEM fails in some sense, because we can have so that and .
Do observe however that here the "or" is being interpreted _externally_, but the "not" is being interpreted _internally_.
Eigil Fjeldgren Rischel
2020
6
12
https://erischel.com/jsd-as-enrichment/
jsd-as-enrichment
/jsd-as-enrichment/
Jensen-Shannon divergence is compositional
Let be the category of finite sets and stochastic matrices.
Given two stochastic matrices, , we can define their
**Jensen-Shannon distance** as ,
where JSD is the Jensen-Shannon divergence.
It's a standard result that the root of JSD defines a metric on the space of
probability measures - hence the above defines a metric on the set
. My aim here is to show that
_this gives an enrichment of in the category
of metric spaces and **short**, i.e distance nonincreasing, maps_
The content of this statement is that the composition map
is a short map. The monoidal structure on
that we're considering is given by the "1-metric", i.e
This has the convenient property that a map is short if and only if it's "short
in each variable separately". In other words, we must show that the map
is short for each , and that the map
is short for each .
Eigil Fjeldgren Rischel
2020
6
12
Fact about .
We will use the following characterization of :
Let be a "fair coinflip", i.e a random variable which is with
probability and otherwise.
Let be a random variable which is distributed according to if and
if . Then the mutual information of and is exactly the
Jensen-Shannon divergence of and
Eigil Fjeldgren Rischel
2020
6
12
Postcomposition
We want to show that .
Clearly we may as well square both sides, so that we're trying to show that
It suffices to show that this inequality holds for each , so let an be
given.
Then is the mutual information between a
fair coin and a variable distributed according to .
and is the mutual information between
and a variable distributed as .
But "postprocessing" by can't possible put more information about into
the random variable, since it depends on only through .
Hence we have the desired inequality.
Eigil Fjeldgren Rischel
2020
6
12
Precomposition
We want to show
Again it suffices to show
It's enough to show this for each specific , so let's assume wlog that
and is simply a distribution, and we're showing
To see this, let be distributed according to , let be an unbiased
coin, and let be distributed according to .
Then is , and
is .
Our claim is that for at least one .
We insert the entropy formula for mutual information:
Since independent, (equals one bit), so we may rearrange
this as
Now that the expected value is precisely by
definition.
By a standard inequality .
Hence we also have the inequality for at least one , as desired.
Eigil Fjeldgren Rischel
2020
6
12
Followup questions
1. Is there a way to build something like this "canonically" - the way entropy
is characterized by a set of axioms in A characterization of Entropy in
Terms of Information Loss?
2. How does this relate to other metrics on probability, i.e the description
from A probability monad as the colimit of spaces of finite samples
Eigil Fjeldgren Rischel
2020
6
2
https://erischel.com/compositionality-for-transfer-learning/
compositionality-for-transfer-learning
/compositionality-for-transfer-learning/
Compositionality for Transfer Learning
_Transfer learning_ is the idea that, after a machine learning system (or a non-machine
learning system, for that matter, like a human) has learned to solve some
problem, it should be able to _transfer_ this knowledge to solving similar
problems. Humans are pretty good at this, at least compared to current ML
systems, which tend to suck.
Why do we expect transfer learning to work? It seems that, in general, we expect
that the solution to a task can be _decomposed_ into several pieces, some of
which will still be useful for the related task.
As an example, suppose we teach a self-driving card to drive to a given place,
using a map of the local area and camera input.
If we could open up the resulting algorithm, we may expect to find "subroutines"
corresponding to
1. Breaking down the visual input into objects.
2. Maintaining/updating an internal memory with this data.
3. Using this data to locate itself on the map
4. Plotting a route on the map between two points
5. Executing a route while driving correctly (i.e not breaking the law, not
causing crashes).
If we now want this system, instead, to locate and follow a car with a specific
license plate (or something), we would expect that most of these routines,
except perhaps 3 and 4, would still be useful. It would not have to learn all
over again how to recognize other cars and involve crashes.
In other words, we expect transfer learning to happen because of
_compositionality_.
To introduce some symbols, we are trying to learn a function from
observations to decisions.
The target behavior is really a function of some high-level description of the
system, as understood by humans, i.e it factors as .
If the system manages to learn as well as , then if we swap out the task
with another one, which is also specified on the same abstract level, , the system will have less to do - a lot of the parameter space
is already in the right configuration.
Of course, if you already knew how to compute the high-level representation
, you mostly wouldn't need machine learning.
However, when we view the problem from this angle, it seems one way to get more
transfer learning is to look for learning algorithms that output "decomposed"
models, so that we can try to separate the abstraction , from the
"task-specific logic" .
One way to do this is to find a class of related tasks, ,
which we feel share a common abstraction.
Then we can pick some set and try to train models - one going
from , the others going from .
We give each of these algorithms the average loss across all the tasks as the
loss. Then the "high-level models" try to make use of the
low-level data as best they can, while the "abstraction" model
tries to create an abstraction which is useful to all the high-level models.
Eigil Fjeldgren Rischel
2020
5
24
https://erischel.com/stochastic-stalks/
stochastic-stalks
/stochastic-stalks/
Stochastic Stalks
Eigil Fjeldgren Rischel
2020
5
24
Stalks and points
Recall that a point of a topos is a geometric morphism from the topos .
My preferred way to think about this is to consider the sheaf topos on
some (sober) topological space .
Then given and a sheaf , we can form the stalk at
1. This determines the point uniquely, i.e if then
2. This is a left exact left adjoint - i.e part of a geometric
morphism .
3. All left exact left adjoints have this form.
Hence it makes sense to identify literal points of with points of in
the above sense.
Eigil Fjeldgren Rischel
2020
5
24
Random points
We can think of a probability measure on a space as a sort of "generalized
point", which has been "smeared out".
How can we lift this intuition to the level of ?
It seems sort of obvious that we shouldn't expect this to work on the level of
sets - they are somehow too "discrete" to capture quantitative information about
probabilities.
It's actually worth mentioning here that any point of is determined by
its action on sheaves represented by open sets, which must each be sent to
either or (this follows from the "left exact left adjoint"
assumptions).
The fact that it's a left exact left adjoint furthermore implies that it must be
a sort of infinitely-additive -valued probability measure defined on
the open sets, and it follows from this that it's a "dirac measure".
This is how to prove that all points are really represented by a point.
It seems that one way of considering "random points" would be to let the functor
take values in a category with objects that can reasonably represent more
complicated probability measures.
It also seems unlikely that we can rely completely on universal properties to
carry the day here.
If are open sets, then is their product, both in
and in .
So if a functor preserves products, depends only
on and - so this clearly can't capture all possible probability
measures.
Eigil Fjeldgren Rischel
2020
5
24
One attempt: Stochastic stalks
I haven't solved this problem completely (I'm not convinced a good general
solution exists).
One approach is to think about _integration of metric spaces over a measure_,
which I will now explain.
A _sheaf of metric spaces_ is a functor , where is the
category of metric spaces and _short_, i.e distance-nonincreasing, maps,
which satisfies the sheaf axiom.
We denote the category of such sheaves by .
By "the sheaf axiom", I mean it preserves limits.
Since does not have all limits, this is a bit subtler than it may appear.
However, since does have finite limits, this difficulty disappears if we
assume is compact.
Let be a sheaf of metric spaces and let be a Radon probability measure
on .
Then we define to be the product of all the stalks
(just considered as a set).
Equip with a pseudometric by setting .
In other words, we _integrate_ the distances according to the given probability
measure. If we quotient out with the relation if , this
gives a proper metric space, .
Now for each with , we consider the map given by taking all the germs.
We let be the metric space consisting of the images of all these maps.
This defines a functor .
We can recover the probability measure on an open set by considering a sheaf
given
by two points at distance if and the singleton otherwise.
Then consists of two points at distance .
At least in the case of compact Hausdorff spaces, this determines the underlying
measure uniquely.
(In general, the "measure defined on open sets" that we recover in this way is
called a valuation, and you can argue that we shouldn't expect to tell the
difference between different measures with the same valuation).
Eigil Fjeldgren Rischel
2020
5
24
Questions
- What useful properties characterize functors of the above form?
- Is there a good way of doing this for -algebras instead of topologies?
- Is there a good way of doing this for a general topos?
Eigil Fjeldgren Rischel
2020
5
24
Example: Random variables
Let be any metric space.
Then we can form a sheaf of metric spaces where is the set of continuous
functions in the sup metric.
Then the metric space is the set of -valued random variables,
metrized by letting - i.e metrized by
_expected_ distance.
Eigil Fjeldgren Rischel
2020
5
17
https://erischel.com/ax-grothendieck-model-theory/
ax-grothendieck-model-theory
/ax-grothendieck-model-theory/
The Ax-Grothendieck theorem
The Ax-Grothendieck theorem says the following:
Let be a polynomial function. If it's
injective, then it's surjective as well.
Here's how to prove it:
1. The statement can be formulated as a first-order statement in the language of
fields
2. If a statement like that fails for , there's a disproof in the
first-order theory of algebraically closed fields of characteristic zero.
3. Such a proof is _finite_, so it only uses finitely many of the assumptions
- hence the theorem also fails in algebraically closed fields of
sufficiently high characteristic.
4. Hence it fails in the algebraic closure of . The specific
counterexample is in some finite extension of , which is a
finite field. But clearly the theorem is _true_ for finite fields, just by counting.
I think this is a pretty cool proof - it uses model theory in a really
surprising way, and the step where you use the fact that _proofs are finite_ is
just totally bonkers.
Eigil Fjeldgren Rischel
2020
5
17
First-order logic and completeness
Recall that first-order logic works with statements built up out of the logical
connectives .
The language of _fields_ augments these symbols with (it's
really just the language of commutative rings - being a field just means that
additional axioms hold).
A statement in the first-order language of fields could be something like this:
This statement says that all nonconstant linear polynomials have a solution,
which is true.
We could also write down a statement like
This says that all fourth-order polynomials have a root, which is not always
true, but true sometimes, like in .
Similarly, we can form a statement for all , saying that
all th order polynomials have a root. Note that the conjunction of all these
statements is _not_ a first-order statement in itself - we can't take
conjuctions of infinitely many statements, and there's no way to quantify over
all polynomials in the theory of fields.
Similarly, we can form statements like . This is also true in
, but fails in fields of characteristic . We can form statements
for all , saying that - that the field does not have
characteristic .
The _theory of algebraically closed fields of characteristic zero_ is the theory
consisting of the axioms of a field augmented with the statements and
for all .
A model of this theory is precisely an algebraically closed field of
characteristic zero (hence the name).
Now here is a highly nontrivial fact about this theory: it is _complete_.
This means that _any statement expressible in the first-order language of fields
is either provable or disprovable in it_. This means that, from the perspective
of first-order logic, there are really no differences between fields of this
type - even though there are of course many different such fields, they have the
same first-order properties.
Eigil Fjeldgren Rischel
2020
5
17
Ax-Grothendieck in first order
We want to phrase the Ax-Grothendieck theorem as a first-order statement.
It turns out this isn't quite possible, because we can't quantify over "all
degree polynomials" or "all ". Therefore we instead
formulate the statement : "Any polynomial function of degree at most is surjective if it's injective".
Then the Ax-Grothendieck statement says that holds for all
(By the way: by "polynomial ", I mean a function
where each coordinate is an -variable polynomial)
How does look as a first-order formula? It's quite complicated - mostly
because a degree polynomial like involves a large number of
coefficients.
The main point is that there is a _definite_ number of coefficients, once
and is fixed, and so we can write "for all choices of coefficients for
..." in first-order logic.
For instance, if , this looks like
Here is the coefficient for in the second coordinate polynomial of
, and so on.
Now of course we can also make sense of injectivity: it's just the statement
- where the equality on the left of the implication is shorthand
for a complicated expression involving the coefficients of .
And we can make sense of surjectivity: it just means
Hence we have all the ingredients to make sense of .
Eigil Fjeldgren Rischel
2020
5
17
Finishing up the proof
We now have most of the ingredients to fill out the proof above.
Since Ax-grothendieck is a (family of) first-order statements, if it fails,
there is a disproof in the first-order theory of algebraically closed
characteristic zero fields.
To be specific, if Ax-Grothendieck is false, then is false for some
specific , and hence there is a disproof of that statement, i.e a proof of
.
Observe that such a proof is finite. Hence it can use at most finitely many of
the statements .
Hence for large enough (larger than any where is used in the
proof), Ax-Grothendieck fails for algebraically closed fields of characteristic
- like the algebraic closure of .
(How do we know it fails? We have a proof that it fails!)
Hence there exists some polynomial function
which is injective and not surjective.
But now we can take the algebraic extension of by all the coefficientsof
and the coordinates of a point which is
not in the image. Call this extension .
Then is a well-defined injective polynomial which is not
surjective.
And since is a finite extension of a finite field, it's finite.
But then we clearly have a contradiction: injective fuctions between finite sets
of the same cardinality are bijections, just by counting.
This concludes the proof.
Eigil Fjeldgren Rischel
2020
5
10
https://erischel.com/jardine-persistent-htpy/
jardine-persistent-htpy
/jardine-persistent-htpy/
Notes from "Persistent Homotopy Theory"
My notes from Persistent Homotopy Theory by JF Jardine.
The goal of the paper is to study "filtered spaces". By this is meant in general
something like an assignment of a "space" or simplicial set to
each nonnegative real .
A prototypical example is the Vietoris-Rips complex of a metric space, .
The idea being pointed towards is some sort of modification of model category
theory to make ideas from persistent homology work more nicely.
The main example considered is an inclusion of VR-complexes
coming from an incusion of datasets where all the points in
are "close to" ponts in .
In this situatio is not generally a homotopy equivalence or
anything like tat, but it's still a bit "equivalency" - we would like to
understand howthis works, and how this plays into classical model category theory.
Eigil Fjeldgren Rischel
2020
5
10
1 Posets
Given a finite subset of a metric space , which we think of as a _data
set_,
we consider the collection of subsets where for all . We can order this by inclusion - it is
exactly the poset of simplices in the Vietoris-Rips Complex .
Now we want to work with this combinatorial data as if it were topological data.
This is generally easiest if we're working with a simplicial set.
We can make this into a simplicial set by choosing an ordering on , but this
is non-canonical.
We can also consider the nerve , but this is somewhat clunky - the
resulting simplicial structure is really the _subdivision_ of the complex
.
( for the nerve of a category - a simplicial set - it somewhat unusual
notation, but I've stuck with Jardine's choice of notation.)
Eigil Fjeldgren Rischel
2020
5
10
2 Stability
Define the Hausdorff Distance on finite subsets of a metric space
as follows:
if and only if for all there exists with ,
and vice versa.
Then a situation of interest is if we have two data sets with "small" Hausdorff
distance - in this case, the datasets seem to reflect mostly the same underlying
topology, so we'd like our methods to give mostly the same results.
What sort of relation do we have between and ?
For simplicity let's work with the case where .
Then we have an inclusion , and we can ask in what sense this
is "equivalence-like".
We can try to cook up an inverse by picking a nearest point
for each .
By assumption .
This means we can build a diagram
The top square commutes - in other words, almost has a retract, except we
have to add an extra error of up to .
The bottom square doesn't commute - after all, does really move around
some points.
But it doesn't move around points too much - the set is still in , i.e: the altered points and the
original points are all within of each other.
The pair of inclusions present a _homotopy_ between and
, so the bottom square commutes up to homotopy.
In other words, and form a sort of "approximate deformation retract".
Eigil Fjeldgren Rischel
2020
5
10
3 Controlled equivalences
In this section we encounter some interesting homotopy theory-ish things.
We consider the category of functors (or other
categories).
We can equip this with the _projective model structure_, meaning a map is a weak
equivalence or a fibration iff it is so sectionwise.
This means the fibrant objects are exactly the functors that land in Kan
complexes, and the cofibrant objects are precisely those that land in
monomorphisms (this is a theorem).
Now the interesting thing we can do is consider various notions of
"-isomorphism".
The basic motivation for this is that, if and , we
get a diagram like this
where the top triangle commutes, and the bottom commutes up to a homotopy fixing
- an "-interleaving"
This tells us that the maps is "almost an isomorphism":
- We can find a preimage for any element in , as long as we're willing
to increase the allowed error by .
- If two elements in agree in , they also agree in
Let's call something like this an -isomorphism.
We can then ask in general for an -equivalence, which gives an -isomorphism on the
(filtered) homotopy groups.
These maps have various good properties
- They are stable under composition with weak equivalences
- They're not quite stable under composition, rather when composing an -equivalence and
an -equivalence, you get an -equivalence.
- They satisfy a similarly modified version of the 2 out of 3 condition.
- A pullback of an -equivalence which is a fibration is a -equivalence
(and a fibration).
- If a map is an -equivalence and a sectionwise cofibration (not the same as
a cofibration in the projective model structure!), it admits a
-interleaving, in the sense of the diagram above.
This gives a sort of bizzarro model structure/homotopy theory for
-equivalences.
In this world, an -interleaving is a bit like a deformation retract.
Eigil Fjeldgren Rischel
2020
5
2
https://erischel.com/complexity-theory-probability/
complexity-theory-probability
/complexity-theory-probability/
Complexity theory, probability
Eigil Fjeldgren Rischel
2020
5
2
Computationally bounded probability theory
Probability theory is about how to manage incomplete information.
One way to interpret a statement like "the probability of event is " is
in terms of betting odds - you think the probability of is if you value
a lottery ticket that pays out $1 if happens at dollars.
From this interpretation, all the laws of probability theory (except arguably
those involving infinite conjunctions of events) follow, if we add the
requirement that your valuation is "inexploitable" - in other words, if we
require that no smart bookie can get you to make a series of bets that always
loses money.
The big issue with probability theory as a model of how to reason under
uncertain information is that it doesn't account for agents with
_limited processing power_ (i.e all of them).
A way to think about this problem is that, realistically, the best you can
possible hope for is to not be outsmarted by bookies without too much more
computing power than you.
For example, what is the probability that among the first digits of
, there are more even than odd ones?
If a mortal human is forced to assign this event a probability, we must choose
more or less blindly - a bookie with an arbitrarily powerful computer could then
win lots of money off us by calculating a lot of inaccessible digits of
and betting us for their parity.
This approach to "computationally bounded probability theory" is explored in the
paper Logical Induction. Based on the idea about "computationally limited
bookies", they build a notion of "logical inductor" - a process which assigns
gradually updating probabilities to logical statements, and which is "optimal"
in the sense that it can't be exploited.
Eigil Fjeldgren Rischel
2020
5
2
Computationally bounded logic
In some sense, the problem solved by logical induction is that ordinary logic is
_too expressive_. It can express statements which are hard to verify in a small
amount of space.
Interestingly, there are versions of logic that get around this limitation in
various ways.
Light Affine Set Theory is a version of set theory with the following
interesting property: the provably total functions are precisely those with a
polynomial-time algorithm.
This means that the function which determines the most common parity among the
first digits of is _not provably total_ (assuming of course
that there is no efficient algorithm for this).
Of course, this sort of logic is by necessity much more restricted than normal
logic.
We can think about this as a sort of ultrafinitism - numbers like
are "too big to exist".
Eigil Fjeldgren Rischel
2020
5
2
Synthesis?
What is the unifying thread here? I'm not sure.
I want to say something like "you can't prove in LAST that a logical inductor is
inconsistent".
This is not quite meaningful - a logical inductor is a _sequence_ of probability
assignments (as they have more time to calculate, they update their
probabilities),
and each step along the sequence will generally contain plenty of specific
verifiable mistakes.
But there's an idea that I'm trying to grasp at, that the inconsistency of a
logical inductor "takes superpolynomial resources to detect", and hence it "is"
a probability measure from the point of view of a polynomial logic.
Or perhaps we should say that it "isn't not a probability measure", in the sense
of constructive logic.
That is, we can't verify that it's a probability measure, but we can't construct
a counterexample either.
Eigil Fjeldgren Rischel
2020
4
26
https://erischel.com/zero-one-laws-paper/
zero-one-laws-paper
/zero-one-laws-paper/
The zero-one laws of Kolmogorov and Hewitt–Savage in categorical probability
Eigil Fjeldgren Rischel
2020
4
26
TLDR
This is a post about my paper The zero-one laws of Kolmogorov and Hewitt-Savage
in categorical probability, joint with Tobias Fritz.
This is a "companion piece" where I try to explain those ideas in a more
understandable language.
There are essentially three ideas in this paper:
- "Markov categories for synthetic probability theory" - this is only treated
briefly, since this is just the background that we're building on top of.
Tobias has a long paper on this, A synthetic approach to Markov kernels,
conditional independence and theorems on sufficient statistics.
- "Kolmogorov products" - a take on tensor products of infinitely many objects.
These are introduced into the framework to handle probabilistic systems with
infinitely many variables.
- "Zero-One Laws". These are theorems from probability theory of the form "If an
event satisfies ..., then " - i.e either it always happens
or it never happens. For two classical ones, Kolmogorov's and Hewitt-Savage's,
we give a way of phrasing them in the categorical language and prove them.
In this post, I'll try to give an accessible take on these ideas. I'll assume
some familiarity with category theory, and a _very_ small amount of familiarity
with probability theory
Eigil Fjeldgren Rischel
2020
4
26
What do you mean, "synthetic probability theory"
Eigil Fjeldgren Rischel
2020
4
26
Synthetic vs analytic
(This is a weird rambly section. Feel free to skip it)
Think about plane geometry for a moment. Let's contrast two approaches:
In the "Euclidean" approach, we work without any reference to what a point, a
line, and so on, _is_[^fn:1]. We simply have certain
axioms which relate lines and points, certain properties (like congruence) which
things can have or not have, and so on. We could also call this approach
_formal_, or maybe _syntactical_.
We can contrast this with how plane geometry might be treated in the modern
world:
- A point is an element of .
- A line is a subset of such that ...
and so on.
Here a point is really a particular _thing_, so is a line, and the statement
"The angle equals the angle " is defined in terms of the things that
are, rather than being a more or less irreducible notion. We call such
an approach _analytical_.
Another way of looking at it is that Euclidean geometry admits several distinct
models (particularly if we abandon certain axioms, e.g. the parallel postulate).
On the other hand, the analytical approach of
augmented with some structure is a _specific_ model.
Eigil Fjeldgren Rischel
2020
4
26
Markov categories
The classical approach to probability theory is analytical - a probability space
is a specific thing with a specific structure.
We want to contrast this to a synthetic approach to probability theory.
One "model" of our synthetic approach will be "classical probability theory",
i.e measures and algebras.
Another model will be given by "possibility theory", where each outcome either
happens or doesn't happen. And there are many other exoctic models like that.
This approach is called Markov categories.
It is a notion which has been developed by several different authors under
different names - see the paper by Fritz above for a review of the literature
(and a much more in-depth treatment of Markov categories).
A Markov category is a symmetric monoidal category ,
where the monoidal unit is terminal, where each object carries a distinguished cocommutative
comonoid structure
and such that the symmetric monoidal structure isomorphisms
are homomorphisms[^fn:2].
See Tobias' paper for an unpacking of this definition.
The interpretation of a Markov category is that the objects are "spaces" that
our random variables can take values in, and the maps are "stochastic processes"
or "kernels", i.e functions which produce their output somewhat randomly.
The tensor product of two objects is the "product space" where points are pairs
of points - but note that this is _not_ a product in the categorical
sense, since, give a random map , we can not in general
recover it from the "marginals" or "projections" and .
This is because the random values of and may be _dependent_.
On the other hand, we do have "diagonal" maps - which
non-randomly send to - and a "projection" map , which is
actually unique (there is precisely one way to randomly generate an element of
the one-point space).
A map is called a _distribution on _, and should be thought of as
the abstract version of a probability measure.
For instance, if is a distribution, and is a map, then
we can make sense of this diagram:
it is the distribution on
where is sampled according to , then is
sampled according to . Note the bullet which denotes the copying operator.
We can define a few different standard terms from probability theory in this
context.
Eigil Fjeldgren Rischel
2020
4
26
Independence
Given a map , we can ask whether are /independent given
/. The intuition here is that given some , if I sample from
and tell you what the was, that gives you no information about the
(given that you know what the was).
In diagram form, this looks like this:
The basic idea here is that it makes no difference whether we generate
together in one go, or separately (using the same ).
Eigil Fjeldgren Rischel
2020
4
26
Determinism
A map is _deterministic_ if we have this equality:
This means if we run twice with the same , we get the same out. This is a reasonable definition of "deterministic".
Note that in both these cases, the existence of the comonoid structure is
_exactly_ what we needed to make sense of the probabilistic notion.
Eigil Fjeldgren Rischel
2020
4
26
Examples
- There is a Markov category where the objects are measurable
spaces, and the maps are Markov Kernels
- There is a Markov category where the objects are finite
sets and the maps are stochastic matrices. It is a full subcategory of
.
- There is a Markov category called where the objects are
sets and the maps are "total relations", i.e relations where each
element is related to at least one element in .
- There is a Markov category where objects are natural numbers,
and maps are "affine maps plus constant Gaussian noise"
For (many) more examples, see Tobias' paper.
Eigil Fjeldgren Rischel
2020
4
26
Infinite tensor products
Most theorems in probability theory involve infinite families of variables.
To make sense of that in our framework, we need to make sense of "distributions
on infinite product spaces", which means we need a notion of "infinite tensor
product".
The definition goes like this: Let be a collection of
objects in a markov category .
For each finite subset , we can make sense of the tensor product
[^fn:3]. Since the tensor unit is terminal, we have natural maps
for each inclusion of
finite sets .
Then we can ask for a cofiltered limit of this system:
This is a reasonable definition of "infinite tensor product",
which we can denote
However, it turns out we need one more condition.
The issue is that in a Markov category, having an object defined up to
isomorphism is not always good enough - we want it up to _deterministic
isomorphism_, so that the comonoid structure is determined as well.
The natural condition to add to make sure we pick the "right" limit is that each
of the projections , for
each finite subset, is deterministic.
Such an infinite tensor product, we called a _Kolmogorov product_.
Eigil Fjeldgren Rischel
2020
4
26
Kolmogorov's Zero-One Theorem, abstractly
With this setup, it turns out it's very easy to prove an abstract version of
Kolmogorov's Zero-One Law.
Informally, the content of the law is this: suppose you have a family of
independent random variables, and some event which is determined by these
variables, but is independent of any finite subset of them. Then
Here is our abstract version:
Let be objects so that the Kolmogorov product exists.
Let be a map, and be a deterministic map.
Suppose displays the conditional independence of the given , and
that for every finite , the joint distribution
displays the independence of and given .
(Here ).
Then the composite is deterministic.
I will sketch the proof (see the paper for details).
Essentially, we first prove that is independent of all the , not just
the finite subsets.
To see this, we note that the definition of independence is comparing two maps
into
.
This is a limit, so it's enough to show that all the component maps, which are
the maps for finite , agree - this is true by definition.
Now it follows by a string diagram manipulation that satisfies the
definition of determinism:
This concludes the proof.
To recover the classical Kolmogorov 0-1 law, we must do a little bit of
technical work.
We work in the Markov category of _standard Borel spaces_.
This Markov category has countable Kolmogorov products[^fn:4]. We let and let
our be the _spaces_ that the variables from the theorem take values
in.
We let the map give the joint distribution of the variables.
We let , and we let be the indicator for the event.
Then the theorem says that the composite is deterministic.
So the probability that is , which is the probability that the event
happens, is either zero or one.
[^fn:1]: Euclid actually did try to define notions like lines, points, etc, but we're gonna use his name anyways
[^fn:2]: Note that we're not assuming all maps, or even all isomorphisms, are homomorphic. This makes Markov categories an evil notion
[^fn:3]: I am handwaving some non-issues with regards to non-strictness here
[^fn:4]: It turns out that proving this is significantly harder than proving the abstract theorem, once you have the setup
Eigil Fjeldgren Rischel
2020
4
13
https://erischel.com/bivariate-causal-inference/
bivariate-causal-inference
/bivariate-causal-inference/
Bivariate Causal Inference
Eigil Fjeldgren Rischel
2020
4
13
TLDR
I give a very short introduction to the idea of "causality" in statistics, then
talk about two ways to infer causal structure for two variables - i.e, without
using conditional independence statements.
The ideas here are mostly taken from Peters, Janzing, and Schölkopf: Elements of
causal inference: foundations and learning algorithms.
Eigil Fjeldgren Rischel
2020
4
13
What is causality?
A "causal statistical model" is something like this:
Or this
Where a classical statistical model tells you what the probability of various
outcomes are, (which you can then use to derive conditional probabilities, etc),
a causal model gives you more information - it tells you how the distribution
changes when the system is intervened on.
For instance, in the first model above, intervening on has no effect on the
distribution of , whereas in the latter, the distribution of
after the intervention is exactly the conditional distribution
.
Eigil Fjeldgren Rischel
2020
4
13
Causal inference by conditional independence
We can actually distinguish between the two causal structures above without
doing interventional experiments.
This is because in the first model, and are independent, but don't
(necessarily) remain independent after conditioning on .
For instance, if , learning that has a certain value tells us
exactly how to compute and from each other.
On the other hand, in the second model, and are not (necessarily)
independent, but after conditioning on , they _are_. That's because the
dependency of on only "goes through" , so if we already know what
is, learning about tells us nothing about .
Why are and independent in the first model? This is because of a basic
principle of causal modeling, which we can paraphrase as "all (conditional or
unconditional) dependencies between variables must be explained by causal
structure".
(Formally, the distribution has the _markov property_ with respect to the
graph).
Eigil Fjeldgren Rischel
2020
4
13
Bivariate causal inference
We can use conditional (in)dependence to try to tell causal graphs apart. But we
can't always do this. For instance, this graph:
Gives rise to exactly the same independence statements as graph 2 above.
(We say they are "Markov equivalent").
The worst case is the case where we have just two variables - since both graphs
and are Markov equivalent.
(And the third possible DAG, , corresponds to the statement that and are independent - not very
interesting).
How can we get around this problem, and detect causal structure when we only
have two variables?
Let's make it concrete, and say that is a variable that measures whether or
not a person smokes, and measures cancer. We observe a strong correlation. We have supposed there are no
confounding variables, but now we still have the question: does smoking cause
cancer, or does cancer cause smoking?
Eigil Fjeldgren Rischel
2020
4
13
Do an interventional experiment
The easiest way is to just test our theory directly against nature.
The difference between the statements and is whether or not
doing the intervention will affect or not.
In other words, we can take a large number of people, randomly assign them to
either smoke or not smoke, then come back a few years later and see whether the
smokers have more cancer.
This experiment may not get past the ethics board.
Eigil Fjeldgren Rischel
2020
4
13
Use domain knowledge
The idea that cancer causes smoking seems pretty dumb.
People generally start smoking, then develop cancer many years later.
We could try to build this idea into a more complicated causal model (with more
variables), which we could statistically test, but we could also just take
seriously the fact that the hypothesis "cancer causes smoking" is prima facie
much less plausible than "smoking causes cancer".
These two solutions are both pretty unsatisfying.
Fortunately, we also have more sophisticated tools.
Eigil Fjeldgren Rischel
2020
4
13
Independent mechanisms.
The basic idea is this: if the causal structure is that smoking causes cancer,
then whatever process turns smoking into cancer is independent of the process
that makes people smoke.
This means that learning the conditional distribution
tells you nothing about the distribution
itself. On the other hand, learning the smoking rates of
cancer victims and non-cancer victims does in fact tell you something about the
cancer rate.
This is called "The principle of independent mechanisms".
The "mechanisms" in question are the distributions and
. They are "independent" in the sense that
each contains no information about the other.
From this principle, we can derive some algorithms for testing the causal
relationship.
Eigil Fjeldgren Rischel
2020
4
13
Semi-Supervised Learning
Suppose we try to train a machine learning algorithm to predict whether a person
has cancer based on whether or not they smoke.
We provide the algorithm with a number of samples , each consisting
of a "smoking?" and a "cancer?" measurement.
We also provide it with a number of _unlabeled_ samples, , with only the
"smoking?" measurement.
This is sometimes called "semi-supervised learning".
Do these extra samples help the algorithm?
The principle of independent mechanisms says "no". These samples only provide
information about the distribution , which gives no
information on the conditional .
On the other hand, if we're trying to predic smoking from cancer, knowing
something about the general cancer rate may actually help.
Thus we can try to distinguish and from each other by seeing
whether adding the unlabeled samples helps or not.
Eigil Fjeldgren Rischel
2020
4
13
Independent noise
Linear models are ubiquitous in statistics. A linear model for , based
on the principle of independent mechanisms, may look like this:
-
-
-
- independent.
In other words, is normally distributed, then is a sum of times a constant and
another normal distribution.
The crucial property of this model is the independence of the noise variables.
If we run a linear regression to predict from , we will get an error term
of (on average) - this is independent of .
But in the other direction, we get with an error term of, on
average, - which is _not_ independent of (since depends on
).
Hence we can find the causal direction by running two linear regressions and
seeing when we get an independent noise term.
Eigil Fjeldgren Rischel
2020
2
28
https://erischel.com/2020-02-28-how-to-make-a-website/
2020-02-28-how-to-make-a-website
/2020-02-28-how-to-make-a-website/
How to Make A Website
Here's what goes into a website.
A server (hardware).
The software which runs on the server, also called a server.
A domain (optional)
An SSL certificate (technically optional but highly recommended)
I will explain what these terms mean, and how to get your own stuff set up.
Please note that this is a guide for people who want to do everything from scratch. There are plenty of easier ways to set up a website. For instance:
Any of a number of blogging systems, like wordpress.com, which will also help you hook up your own domain.
nearlyfreespeech.com - an extremely cheap option if you want to do some things yourself. If your website is just a collection of static pages, this is an easy way to put it on the internet.
Eigil Fjeldgren Rischel
2020
2
28
How the internet works
When you type "reddit.com" into your browser, here's what happens:
- Your computer sends a signal to something called a "DNS server", saying "I need to find 'reddit.com'. Can you tell me where it is?".
- Since the people who operate reddit have previously left their address at the DNS server, it says, "yes, you can find it at 151.101.193.140".
- Then your computer sends a new message to 151.101.193.140, saying essentialy "I'd like your home page, please"
- (if you'd typed in "reddit.com/r/stuff", it would say "I'd like what you have at '/r/stuff', please").
- Your ISP uses those numbers to find the reddit server, which is just a computer running in a warehouse somewhere.
- The reddit server sends a message back containing their home page
- Your browser displays the page.
Now, what we want to do is the same thing that reddit is doing. So, let's go over the list of things again:
- We need a __domain__, a name for people to find the website. In principle, you can just tell people an IP address like 151.101.193.140, and not use a domain, but that's kind of inconvenient to remember.
- Basically, to get a domain, you pay the people in charge of domains some money for ownership of it. Then they write down in the big list of domains that "awesome-website-stuff.xyz" belongs to so and so, and that the IP address is such and such.
- (In reality, you buy your domain from an intermediary organization, of which there are many)
- We need an actual server, to send the message to people who ask for it. A server is really just a computer - you could "serve" your website from your desktop computer, if you wanted to. In practice, you'll want your server to be running all the time, so it'll probably be easier to rent one. There are many services to do this.
- Then you'll need to run a program on the server which can interpret the messages from people who want to see your website and respond with the website. Confusingly, this program is often also called a server.
- Lastly, for security reasons, you may want your website in HTTPS. This makes sure the contents of your site are secure. If you'll be sending and receiving information which is not public, this is absolutely essential. Even if you just have a public-facing website with nothing secret on it, setting up HTTPS is a good idea - for instance, it ensures that an attacker can't fake messages from your website, so people can trust that it's really you writing what you put on your website.
Before we proceed, I should mention that I AM NOT A SECURITY EXPERT. If you're going to be handling any sort of sensitive information, please, PLEASE, do some research, and try to consult with someone who knows what they're doing. If you only follow my advice, you should assume that anything you put on your server is immediately compromised.
Eigil Fjeldgren Rischel
2020
2
28
Server (hardware)
Basically, we're going to rent a computer from someone for your website to run on. There's a bunch of services to do this, each with their own advantages and problems.
My personal website uses Linode. Their servers start at $5/month for a so-called "nanode". When I need to explain something, I'll use that as the example.
I won't say that I have any great familiarity with the market, so do your own research. The important point for this tutorial is that you get full root access to a server running something like Ubuntu linux.
A lot of services "manage" you in various ways - providing a standardized webserver (as in, __software__ running on your server), limiting the stuff you can do on the server, etc etc. These are not necessarily bad ideas, but it's not what we're looking for.
You can also run whatever OS you want on your server.
You'll almost certainly want some variation of Linux. I'll use Ubuntu for this guide. If you're using a different version, things will obviously be different.
- Buy a server somewhere
- Set it up to run a recent version of Ubuntu.
- Make sure you know the root password of your server (you should be able to view or set this on the website of your provider)
- Then find its IP address.
- Here's how this looks on Linode:
You can see the IP addresses on the right.
- Do `ssh root@x.y.z.w`, substituting the actual IP address
- Log in using the root password
Now you're running a terminal on your server.
Eigil Fjeldgren Rischel
2020
2
28
Server (software)
We're gonna use a program called nginx to serve our website.
- While `ssh`ed into your server, do `apt update` and `apt install nginx` to install nginx.
- To start nginx, do `/etc/init.d/nginx start`
- To test that it's working, open up a browser and type your ip into the address bar. Your should see a page like this:
Now, I'll explain how to configure nginx to serve your website.
- For this basic tutorial, we'll just set up nginx to serve some .html files in a directory. For a more complicated website, you'll probably want __dynamic__ content, i.e content which is generated by a program running on your server, rather than just read from a file. You can set up nginx to do this, but I won't cover that.
- By default, nginx uses the directory `/var/www/` for websites. We're gonna stick with that. Now would be a good time to make up a domain name for your website. Do `mkdir -p /var/www/example.com/html`
- Create `/var/www/example.com/html/index.html` using `nano` or another editor. For now, just put this text in it:
Welcome to example.com!
Success! The example.com server block is working!