Eigil Fjeldgren Rischel 2024 4 25 https://erischel.com/lcc-001R/ lcc-001R /lcc-001R/ and Definition Let be the category where objects are pairs consisting of a convex space and a convex function, and where morphisms are affine maps so that . Let be the category where objects are pairs consisting of a convex space and a concave function, and where morphisms are affine maps so that . References Context Eigil Fjeldgren Rischel 2024 4 30 https://erischel.com/lcc-001Z/ lcc-001Z /lcc-001Z/ Convex Duality made Difficult Eigil Fjeldgren Rischel 2024 4 30 Introduction The study of convex functions - in particular, of their optimization (really minimization) is one of the most important fields of applied mathematics. Convexity seems to be one of those incredibly well-chosen hypotheses which is just specific enough to admit a wealth of theorems, just general enough to produce a nontrivial theory (and a large amount of important examples). Convex optimization, possibly because it has an "analytical" rather than "algebraic" feel, has not been very thoroughly studied by applied category theorists. The one notable exception is Reference , which studies the decomposition of optimization problems by categorical means. This paper takes a different approach, attempting to define a category with optimization problems as the objects, and to derive theorems about optimization by categorical means. Eigil Fjeldgren Rischel 2024 4 30 Convex optimization Eigil Fjeldgren Rischel 2024 4 22 https://erischel.com/lcc-001G/ lcc-001G /lcc-001G/ Standard form convex optimization problem Definition A convex optimization problem in standard form consists of A convex function A list of convex functions A list of affine functions The problem then is to find which minimizes subject to the constraints Eigil Fjeldgren Rischel 2024 5 17 https://erischel.com/lcc-002K/ lcc-002K /lcc-002K/ Lagrangian of an optimization problem Definition Let be a standard-form convex optimization problem, as in Definition . Then the Lagrangian of this problem is the function defined by Recall that denotes the nonnegative reals. Observe that is if satisfies the constraints of the problem, and otherwise. Hence we can think of this minimization problem as playing a zero-sum game: we choose , our adversary chooses , and our loss function is . It is natural to ask about the existence of Nash equilibria in this game - observe that the existence of an equilibrium means that . This is of great utility in solving the original problem. The dual problem is the problem of maximizing the function . This is always a concave problem. In the world of convex optimization, two problems whose constraints carve out the same subset of (and where the function to optimize is the same) would be called equivalent. But they can clearly not be regarded as isomorphic, because the choice of constraint functions makes an important difference to the theory of optimization (for example, it can lead to different dual problems). Here we take the viewpoint that the Lagrangian is really the fundamental object in convex optimization - by passing to a suitable category of Lagrangians, we can make the dual problem into an actual self-duality on this category. Eigil Fjeldgren Rischel 2024 4 30 Convex spaces Eigil Fjeldgren Rischel 2024 4 30 https://erischel.com/lcc-0020/ lcc-0020 /lcc-0020/ Convex Space Definition The category of convex spaces is the category of algebras for the monad of discrete finite-support distributions. The morphisms are called -homomorphisms or homomorphisms of convex spaces. So as not to multiply notation unnecessarily, we simply denote the category of convex spaces by , using the usual notation for the Eilenberg-Moore category. Eigil Fjeldgren Rischel 2024 5 20 https://erischel.com/lcc-002N/ lcc-002N /lcc-002N/ Definition A function between vector spaces is called affine if it preserves those linear combinations where Eigil Fjeldgren Rischel 2024 5 20 https://erischel.com/lcc-002M/ lcc-002M /lcc-002M/ Definition Let be a convex space. A convex function on is a function so that A concave function is a function so that is convex (in other words, satisfies the opposite inequality). The term "convex function" in this sense clashes with the usual practice of naming structure-preserving functions after the structure they preserve (since convex functions do not preserve the convex structure). Unfortunately this usage is far too established to alter. (Convex functions are called convex because they are exactly those functions where the area above their graph is a convex subset of . Although there appears to be no particular reason why the terms convex and concave should not be interchanged, other than convention). Eigil Fjeldgren Rischel 2024 5 19 https://erischel.com/lcc-002R/ lcc-002R /lcc-002R/ Jensen's Inequality The inequality which holds whenever is convex, is called Jensen's inequality. Sometimes this name is used for a stronger version of this inequality, like the claim that if is a random variable valued in the domain of . These generally follow just from convexity of . Eigil Fjeldgren Rischel 2024 5 19 https://erischel.com/lcc-002Q/ lcc-002Q /lcc-002Q/ There is an natural way to extend the convex structure of to both and , by the convention that any nontrivial convex combination involving an infinity is equal to that infinity. This also gives the adjectives convex and concave a meaning when applied to functions . For example, a function is convex if and only if the subset where it's finite is a convex subset of , and it's a convex function in the ordinary sense on this set. This doesn't work for the extended real line , since there is no sensible interpretation of . We will inescapably meet some functions which take value in the full extended reals, but where we still wish to speak of their convexity (or concavity). Hence we adopt the convention that a function is convex if it obeys Jensen's inequality whenever it makes sense, i.e whenever we do not have or vice versa. Eigil Fjeldgren Rischel 2024 5 20 https://erischel.com/lcc-002O/ lcc-002O /lcc-002O/ Proposition A function between vector spaces is affine if and only if it is a -homomorphism. Eigil Fjeldgren Rischel2024520Proof It's clear that an affine function is a -homomorphism. Suppose is a -homomorphism. Note it suffices to prove preserves binary affine combinations (for not necessarily in ). If , we are done by assumption. Otherwise suppose 1]]> (if not, replace it by by symmetry). Then This is a convex combination, so Rearranging, we find as desired. Justified by Proposition , we will appropriate the term affine to refer to -homomorphisms, even between convex spaces which are not vector spaces. There is generally no chance of confusion, but it's worth emphasizing that the use of this term does not entail that the domain is closed under arbitrary affine combinations, for example. Convex spaces admit both a Cartesian product (given by the product of the underlying sets equipped with pointwise operations) and a tensor product, which (co)represents "bihomomorphisms". This is analogous to the situation for vector spaces. Unlike vector spaces, however, since all constant maps are homomorphisms, the projections are bihomomorphisms, which induces a map . Thus homomorphisms are a subset of bihomomorphisms. Since we are generally dealing with convex or concave functions, which can't freely be extended to the tensor product, we will work with the Cartesian product in this paper. But it's very possible that most of our constructions would work also with the tensor product, and maybe there is some situation where the extra generality is necessary. Eigil Fjeldgren Rischel 2024 5 21 https://erischel.com/lcc-002S/ lcc-002S /lcc-002S/ Simplex Definition The free convex space on a finite set of elements is called the -simplex and denoted (the reason for the apparent mismatch of numbering is that the -simplex is -dimensional). Note that an element of is a tuple so that and . In particular, . Eigil Fjeldgren Rischel 2024 5 21 https://erischel.com/lcc-002U/ lcc-002U /lcc-002U/ Topological convex space Definition A topological convex space is a convex space equipped with a topology so that any affine map is continuous (when is given the subspace topology). Eigil Fjeldgren Rischel 2024 4 30 The Category of Minmax problems Eigil Fjeldgren Rischel 2024 4 22 https://erischel.com/lcc-001C/ lcc-001C /lcc-001C/ Minmax problem Definition A minmax problem is a triple where are convex spaces (that is, algebras for the discrete distribution monad---more topological assumptions may be necessary here), and is a function which is Pointwise convex in ---for each , given , Pointwise concave in ---for each , given , A morphism of minmax problems is a pair of functions and so that We will see that various constructions on this category, which are natural and well-behaved from the point of view of category theory, capture relevant constructions from the theory of convex optimization. is bifibred over , and the Cartesian and coCartesian lifts capture the operations of minimizing over the primal variables or maximizing over the dual variables The property of strong duality amounts to the claim that a particular diagram has the local Beck-Chevalley property Relatedly, the existence of a Nash equilibrium for the game corresponding to amounts to the existence of a certain morphism. The fact that this implies strong duality can be derived by purely categorical means. Eigil Fjeldgren Rischel 2024 5 10 https://erischel.com/lcc-002H/ lcc-002H /lcc-002H/ Proposition Let be a minmax problem. Suppose is a convex subspace of a vector space , and is affine for each . Then there exists functions , so that . Observe that minmax problems affine in are thus very similar to standard-form convex optimization problems, the main difference being that the set of allowed points in may be constrained in some other way than by requiring certain coordinates to be nonnegative. On the other hand, if is thus constrained, the proposition doesn't actually imply that are convex! The easiest way to see this is by considering for some nonzero . Then we have , and clearly we can choose this decomposition in such a way that these functions are not convex. However, if contains the positive cone for example, we do have both and all the coordinates of convex. Eigil Fjeldgren Rischel 2024 4 22 https://erischel.com/lcc-001D/ lcc-001D /lcc-001D/ Primal and dual optimization problems Definition Let be a minimax problem. The primal optimization problem associated to is the function (the problem being to minimize this function). The dual optimization problem is the function Eigil Fjeldgren Rischel 2024 4 23 https://erischel.com/lcc-001J/ lcc-001J /lcc-001J/ Dual minmax problem Definition Let be a minmax problem. Then let denote the dual problem given by . If is a morphism of minmax problems, then is again a morphism in the other direction. This assignment makes into a self-inverse functor on the category of minmax problems We will often utilize this duality to abbreviate proofs, proving something, for example, for the forwards direction and arguing "by duality" that it holds for the backwards direction as well. Eigil Fjeldgren Rischel 2024 5 1 https://erischel.com/lcc-0027/ lcc-0027 /lcc-0027/ Backwards and forwards morphisms Definition Let a morphism in be called forwards if is an isomorphism, and backwards if is an isomorphism. Let denote the set of forwards morphisms, the set of backwards. Then clearly form an orthogonal factorization system - in fact, both and do. Note that consists exactly of the local equivalences for the inclusion of , so that the localization of can be formed as the terminal forwards map from it, which is clearly (of course, this is not surprising). We will say a morphism in is forwards, respectively backwards, if it is so considered as a morphism in , and reuse the notation for these subclasses of morphism. Eigil Fjeldgren Rischel 2024 5 7 https://erischel.com/lcc-002A/ lcc-002A /lcc-002A/ Lemma Let be convex spaces and let be a convex subspace. Let be a convex function. Then is again convex. Eigil Fjeldgren Rischel202457Proof Let be given, and consider: Since if then , we have that this is less than: because in the latter we are taking the infimum over a smaller set of 's Applying convexity, we get This is precisely the desired inequality. Eigil Fjeldgren Rischel 2024 4 30 https://erischel.com/lcc-0023/ lcc-0023 /lcc-0023/ Proposition The forgetful functor is a bifibration. Moreover, we have the following description of the (co)Cartesian morphisms over backwards and forwards maps. A forwards morphism is Cartesian if and only if for all A forwards morphism is coCartesian if and only if A backwards morphism is Cartesian if and only if A backwards morphism is coCartesian if and only if Eigil Fjeldgren Rischel2024430Proof Note that it suffices to provide Cartesian and coCartesian lifts for backwards and forwards morphisms (Definition ), since such lifts compose. Hence it suffices to verify that the given descriptions are correct, since clearly they suffice to compute a (co)Cartesian lift over any such morphism. Note also that, since the forgetful functor is faithful, to verify a morphism is (co)Cartesian, it suffices to prove that any factorization in the base lifts - uniqueness is automatic. Thus let be so that . Note that composition of a -homomorphism with a convex function is again convex, so this is indeed an object of Now let be some morphism so that we have the factorization in . The goal is now to prove is a homomorphism. This is the inequality which holds by assumption Let be as above, but suppose . First, observe that by lcc-002A, this function is in fact convex in as desired. Let be given, and now suppose we have a factorization in the base. We must prove that but since this amounts to the equation which is again true by assumption. Now the case for backwards morphisms simply follows by duality. What's "really" going on here is that is a two-sided fibration, the result of taking the functor carrying a pair to the poset of minmax problems (in the opposite order), with morphisms acting by precomposition, and applying the Grothendieck construction "contravariantly in the first variable and covariantly in the second variable". (And then observing that the precomposition action has left/right adjoints given by /, to make this into a bifibration). But the theory of two-sided fibrations is quite complicated in general, and we will not go into it here. Note also that this functor is quite close to displaying as topological. If we remove the restriction that minmax problems be convex/concave, we can construct the universal lifts required using a similar supremum formula. The problem is that the supremum of a general set of concave functions is not automatically concave (however, the supremum taken over a convex set, in a suitable sense, is). Eigil Fjeldgren Rischel 2024 4 25 https://erischel.com/lcc-001R/ lcc-001R /lcc-001R/ and Definition Let be the category where objects are pairs consisting of a convex space and a convex function, and where morphisms are affine maps so that . Let be the category where objects are pairs consisting of a convex space and a concave function, and where morphisms are affine maps so that . Eigil Fjeldgren Rischel 2024 4 25 https://erischel.com/lcc-001S/ lcc-001S /lcc-001S/ Proposition The assignment defines a functor Similarly, defines a functor . (The reason for this idiosyncratic way of writing a contravariant functor will become apparent in a minute) The assignment defines a functor (identity on morphisms) , and vice versa. Then The assignment defines a fully faithful functor , whose essential image consists of those tuples where is singleton. Analogously, defines a fully faithful functor We will abuse notation and identify and with their images under these inclusions - thus, for example, will be regarded as an object of . is right adjoint to the inclusion of , and (viewed as a functor ) is left adjoint to the inclusion of Using these identifications, we have Note that if is a morphism of , the two meanings of the notation agree, and the same is true of . Note also that the reflexive subcategory is the local subcategory with respect to the forwards morphisms - a morphism is forward if and only if is an isomorphism (by definition), and the unit is the terminal forwards morphism with domain . A dual statement holds for (it is the colocalization with respect to the class of backwards morphisms). Eigil Fjeldgren Rischel 2024 4 23 https://erischel.com/lcc-001N/ lcc-001N /lcc-001N/ Monoidal structure on minmax problems Definition There is a monoidal structure on minmax problems, given by The unit here is . A state is a point so that for all . More interesting is asking for a state of . This is a pair so that the inequality holds for all Note that for all , this is the minmax inequality (or "weak duality"). Thus a choice of giving a state gives equality in that inequation---it is a solution of the minmax game. In other words, has a state if and only if strong duality holds for , and the state is given by an optimal and dual optimal pair in that case. (By duality, and since , such an object has a state if and only if it has a costate) Eigil Fjeldgren Rischel 2024 5 8 https://erischel.com/lcc-002D/ lcc-002D /lcc-002D/ Proposition The forgetful functor is a monoidal fibration, in the sense of Reference , (see also Reference ). It is also a monoidal opfibration - in other words, both the classes of Cartesian and coCartesian maps are stable under tensor product. In the case of a Cartesian base, a monoidal fibration (like the one we have here) is equivalent to a fibration with a monoidal structure on each fiber, compatible with the reindexing in a certain way. Our base is not Cartesian, but does seem to come from a monoidal structure on each fiber, given by addition of s. The point is that, given , there is a canonical way to obtain an on , given by using a Cartesian lift of and a coCartesian lift of . This suggests there should be a useful theory of monoidal two-sided fibrations, but this notion does not appear to have been studied before. Eigil Fjeldgren Rischel 2024 5 19 https://erischel.com/lcc-002P/ lcc-002P /lcc-002P/ Proposition The localization (resp. colocalization) () is monoidal, in the sense that the class local equivalences is stable under tensor products. The thus induced monoidal structure on is given by (and the same for ). In particular the (co)localization functor is strong monoidal. Eigil Fjeldgren Rischel 2024 4 30 Strong duality Eigil Fjeldgren Rischel2024426https://erischel.com/lcc-001W/lcc-001W/lcc-001W/weak dualityProposition Let be a minmax problem. Then where we abuse notation by identifying a minmax problem with the number Eigil Fjeldgren Rischel2024426Proof The equations are clearly true by definition. Note that the inequality is equivalent to the existence of a morphism We have canonical morphisms Since is a right adjoint to the inclusion of , the above composite induces a map Since is a left adjoint to the inclusion of , that map induces a map as desired. Eigil Fjeldgren Rischel202459https://erischel.com/lcc-002F/lcc-002F/lcc-002F/Strong dualityDefinition Let be a minmax problem. By Proposition , there is a morphism . We say satisfies strong duality if it is an isomorphism. (Note that this is really just an inequality of real numbers, which must be an equality). Eigil Fjeldgren Rischel2024426https://erischel.com/lcc-001X/lcc-001X/lcc-001X/Proposition Let be a minmax problem. Suppose there exists . Then strong duality holds, i.e gives a morphism Here we use the isomorphisms and vice versa, as well as strong monoidality of and . The existence of that morphism means that which is the other direction of the morphism we wanted. If a minmax problem is a zero-sum game, a point is a choice of Nash equilibrium for this game. Eigil Fjeldgren Rischel202456https://erischel.com/lcc-0029/lcc-0029/lcc-0029/Proposition Let be a minmax problem. Then there is a canonical commutative diagram in which is a pullback. obeys strong duality if and only if this square has the local Beck-Chevalley condition for , in the sense that the canonical map is an isomorphism. Eigil Fjeldgren Rischel202456Proof Recall that . Hence the claim is just that which is precisely strong duality. (For more on the Beck-Chevalley condition, see Reference or nlab: Beck-Chevalley Condition)Eigil Fjeldgren Rischel2024518https://erischel.com/lcc-002L/lcc-002L/lcc-002L/Slater's Constraint QualificationProposition Consider an optimization problem in standard form: Minimize Subject to And With each convex and each affine Now assume that is such that are all affine (possibly , and none of the are affine), and suppose there exists so that for all l]]>, and for all other , for all . Then strong duality holds for this optimization problem, and the dual optimal value is attained. Note: This is usually stated for a function defined on an arbitrary convex subset of . In this case we must further ask that is in the relative interior of this domain. Eigil Fjeldgren Rischel2024518Proof (Proof adapted from ) For simplicity, we will assume , i.e we will not assume any of the are affine, and for all . (This is the standard form of Slater's constraint qualification). Let be a matrix and vector so that . Then assume without loss of generality that has full rank. (Suppose is in the span of the other s. Then if the affine constraints are feasible at all, the equation corresponding to must be a consequence of the others. Hence we can delete without altering the primal optimal value, and given a dual optimal value for the problem with deleted, just set .) Let Observe that the optimal value is . Let . It's not hard to see that these sets are disjoint and convex, and hence there exists a hyperplane separating them. In other words, there exists (not all ) so that By the first inequality, we must have (or the right-hand side would be unbounded below on , which is impossible). Similarly we have . The latter of the two statements is equivalent to the statement that when , which simply means . Combining this with the first statement, we get for all First assume . Then we can divide out and get , which proves that is a dual optimal value and that strong duality holds. If , we have for all Inserting , we find and since we must have . This then implies for all . But since this is an affine function, this can only be true if it's constantly zero. Since is nonzero and has full rank, this is impossible. Eigil Fjeldgren Rischel2024422https://erischel.com/lcc-001E/lcc-001E/lcc-001E/Minimax theoremTheorem Let . If are both convex, compact subspaces of finite-dimensional vector spaces, and is continuous, then strong duality holds for , and moreover an equilibrium exists. This theorem can be derived from the Kakutani fixpoint theorem in a very similar way to the usual proof of Nash's theorem about general, non-zerosum games - although note that it is not a special case, since and may not be simplices, and the payoff function here is merely convex, not necessarily affine as it is for a game-theoretic game. However, we will give a different proof, which uses the structure of in a more direct way. Essentially, we will use compactness to reduce to the case of simplexes, then use an inductive argument to reduce to the case where which can be shown by a direct topological argument. The inductive step is a fiber sequence argument, where we use the characterization of strong duality in terms of the Beck-Chevalley property, Proposition . Eigil Fjeldgren Rischel2024521https://erischel.com/lcc-002T/lcc-002T/lcc-002T/Solvable pairDefinition Let be topological convex spaces. We say the pair is a solvable pair if, for any continuous minmax problem , strong duality holds. Eigil Fjeldgren Rischel2024521https://erischel.com/lcc-002V/lcc-002V/lcc-002V/Proposition The pair (in other words, ) is solvable. Eigil Fjeldgren Rischel2024521Proof Let be a continuous minmax problem. Suppose strong duality does not hold. Then by adding a constant to , we can arrange that Consider the set 0\}]]>. Since we must have 0]]> for each , the first projection must be surjective. Since each fiber is convex, and hence connected, and the projection is open, is connected. As an open connected subset of a convex space, it is path connected. Hence there exists some path where and . In other words (picturing the square with the first coordinate horizontal), there exists a path from the left to the right side of the cube so that 0]]> everywhere on the path. Dually, there also exists a path from top to bottom so that is strictly negative everywhere on that path. But they must intersect somewhere, and this is a contradiction. Hence must have a state or a costate, finishing the proof. Eigil Fjeldgren Rischel2024521https://erischel.com/lcc-002W/lcc-002W/lcc-002W/Proposition Let be a continuous, affine map between topological convex spaces. Let be another topological convex space, and suppose is solvable. For every , is solvable, where is the fiber. Then also is solvable. Eigil Fjeldgren Rischel2024521Proof Recall that being solvable means the following square has the Beck-Chevalley condition for continuous : Now we can factor this as follows: Note that the right-hand square here has the Beck-Chevalley condition by assumption. So it suffices to show the left-hand square does. For a given , this means showing that these two functions on are the same But this equation, for some given , is exactly strong duality in the restriction of to , which must hold because this is a solvable pair by assumption. Eigil Fjeldgren Rischel2024521Corollary If is solvable, so is for each , because the map which picks out the first coordinate has fibers isomorphic to so we can proceed by induction (the case being trivial.) Eigil Fjeldgren Rischel2024521https://erischel.com/lcc-002X/lcc-002X/lcc-002X/Lemma Let be a compact topological convex space. Suppose is solvable for all . Then is solvable for all topological convex spaces . Eigil Fjeldgren Rischel2024521Proof Suppose for contradiction does not have strong duality, and assume without loss of generality that . For each let consist of those so that . Given some finite family consider the induced map and apply solvability to the problem - this implies in particular that . This means the family has the finite intersection property, so by compactness it has nonempty intersection. But then an element of the intersection must satisfy which is a contradiction. Eigil Fjeldgren Rischel2024521https://erischel.com/lcc-002Y/lcc-002Y/lcc-002Y/Corollary If is compact, is solvable for any . Eigil Fjeldgren Rischel2024521Proof By Proposition and its corollary, applied to Proposition , we have solvable for all . Since is compact, this means is solvable for all , by Lemma . Now by duality we have solvable for all which means is solvable, and by using Lemma again, we're done. Eigil Fjeldgren Rischel2024430 By Corollary , the pair is solvable, and strong duality holds. Since are both compact, there must exist attaining the infimum and the supremum . These form an equilibrium. It is interesting to note the use of compactness here. Recall that topological compactness is closely connected with the property, also called compactness, of preserving filtered colimits (this property, instantiated in , is not actually the same thing as topological compactness). Our use of compactness here, to derive from the existence of a state in the "finitary" subproblems the existence of a state in the entire problem, does not have this form (nor is it even the case that is the colimit of its subsimplices), but it's possible that the proof could be rewritten to make this step more categorical. The idea of proceeding by induction on was inspired by Reference , although our proof is rather different - they are only looking at affine games, and hence their induction step is completely different (and they have no need for the complicated base case that we do), and since we are not merely interested in games on simplices, we need an additional compactness argument. We can use the minimax theorem to derive other statements of interest about convex optimizationEigil Fjeldgren Rischel2024522https://erischel.com/lcc-002Z/lcc-002Z/lcc-002Z/The separating hyperplane theorem (compact case)Theorem Let be disjoint, compact, convex subspaces. Then there exists and so that whenever . Eigil Fjeldgren Rischel2024522Proof Consider the minmax problem Since the closed unit ball is compact, by the minimax theorem there exists an equilibrium , which then satisfies By disjointness, must be nonzero, so with a suitable choice of we can clearly make the left-hand item strictly positive. Hence 0]]>. Now there must exist some so that 0]]>. By the equilibrium property, we see that must minimize on , and analogously must maximize on . Hence for all , we have which concludes the proof. Eigil Fjeldgren Rischel2024522https://erischel.com/lcc-0030/lcc-0030/lcc-0030/The separating hyperplane theorem (general case)Theorem Let be disjoint convex subsets. Then there exists so that for all . Eigil Fjeldgren Rischel2024522Proof Let be two sequences of sets with the following properties: For each , are disjoint. For each , Each of the are compact and convex These can be constructed for example by taking the intersection of and with the boxes to obtain compact, convex, disjoint subsets which exhaust and . Now apply Theorem to obtain a sequence of so that is negative on and positive on . By compactness of the unit ball, this sequence has a point of density . Now for every pair , we can find some so that is within an arbitrary of and the same is true for , and so that . But then is within of which is positive, so that . Now for each , has a maximizer on and a minimizer on . Hence, by an argument analogous to the proof of Theorem , there is a nonempty closed interval so that, if we have on and on . But since the sets are increasing this sequence of intervals must be decreasing, and hence the intersection must be nonempty - and then any in this intersection will make nonpositive on , nonnegative on , as desired. Eigil Fjeldgren Rischel 2024 4 30 The Legendre Transform Eigil Fjeldgren Rischel 2024 4 25 https://erischel.com/lcc-001P/ lcc-001P /lcc-001P/ Convex conjugate Definition Let be a (real) vector space, and be a function (not necessarily linear). Then the convex conjugate is defined by The convex conjugate is also called the Legendre transform or the Fenchel-Legendre transform. It is intimately related to convex duality. We will prove the following fundamental property of the convex conjugate using the categorical language of minmax problems, and along the way we will see the role that convex duality plays. Note that our invocation of the term "strong duality" here is somewhat more complicated than strictly necessary - normally one would merely invoke the separating hyperplane theorem directly. Eigil Fjeldgren Rischel 2024 4 25 https://erischel.com/lcc-001Q/ lcc-001Q /lcc-001Q/ Proposition Let be convex, so that is a minmax problem. Then we can form the modified minmax problem - note that, up to a sign change in the domain, this amounts to adding the constraint . Then Note that the two uses of the asterisk in this equation conflict. We have both the reversed optimization problem given by flipping the variables, and the convex conjugate function . It may be good to alter this notation to resolve the conflict. Eigil Fjeldgren Rischel 2024 4 25 https://erischel.com/lcc-001T/ lcc-001T /lcc-001T/ Proposition Given a minmax problem where is a finite-dimensional real vector space, let Note that when and otherwise. Thus this amounts to adding a constraint that . Analogously, define Then the Legendre transform (viewing both and as minmax problems using the inclusion ) Eigil Fjeldgren Rischel 2024 4 25 https://erischel.com/lcc-001U/ lcc-001U /lcc-001U/ Proposition Let be a continuous convex function defined on a vector space. Then there is strong duality in the minmax problem Eigil Fjeldgren Rischel2024425Proof Observe that is a closed convex set. Hence there is a hyperplane through so that the entire set is in one half-space. This means a nontrivial affine equation which is satisfied whenever , and where . Clearly for some . If we have for all , which impossible. So by normalizing let's set . This means . Recall that the minmax problem is given by . Strong duality means . We always have the inequality , so it suffices to identify an so that Clearly, for our , we have , since the supremum is unless . On the other hand, taking , we have for all by construction, finishing the proof. Eigil Fjeldgren Rischel 2024 6 4 https://erischel.com/lcc-003C/ lcc-003C /lcc-003C/ Lemma Given a commutative square: in with the Beck-Chevalley property for the fibration from let be some other pair of convex spaces. Then the square also has the Beck-Chevalley condition Eigil Fjeldgren Rischel 2024 4 25 https://erischel.com/lcc-001V/ lcc-001V /lcc-001V/ Proposition Let be a convex function. Then (where denotes the Legendre transform), under the identification of a finite-dimensional vector space with its double dual. Eigil Fjeldgren Rischel2024425Proof Recall that as a minmax problem. Then the claim is that Using first the rewrite and the notation we can rewrite that as Now observe that, restricted to the subcategory given by minmax problems where are real vector spaces, and those homomorphisms given by linear (rather than merely affine) maps, and form endofunctors, and . Since is left adjoint to the inclusion, we have . Hence there is a canonical map, the unit of the adjunction, for any . If is an element of , then by the universal property, this map factors over . This gives us the inequality . (Note that this inequality actually holds even if is not convex, and indeed we haven't really used convexity yet). Observe that, using the natural identification , we have Clearly since the supremum is infinite unless . But observe that Our claim now is that we may exchange these extremizers by strong duality. This amounts to the claim that the local Beck-Chevalley property holds for this square at : But by Proposition , strong duality holds in every square of the form and by Lemma , this is establishes that the previous square has the Beck-Chevalley condition as well, which finishes the proof. Eigil Fjeldgren Rischel 2024 4 30 Misc stuff (sorting) Eigil Fjeldgren Rischel 2024 4 26 https://erischel.com/lcc-001Y/ lcc-001Y /lcc-001Y/ Composition of minmax problems Let be a real vector space, and let be minmax problems. Then we can try to define a composite minmax problem For this composition to be associative relies on a strong duality property. We probably shouldn't want to treat this as well-defined unless it holds. Note that by the convex duality stuff, the minmax problem acts as an identity for this composition. This may fit together into a double category type structure for minmax problems (maybe restricted to linear maps between the spaces). Eigil Fjeldgren Rischel 2024 4 23 https://erischel.com/lcc-001M/ lcc-001M /lcc-001M/ Minmax problems are not star-autonomous Since the category of minmax problems is very similar to a Chu construction, we might hope that we could define a similar star-autonomous structure on minmax problems. Unfortunately, this does not work. We do have the duality, Dual minmax problem, but it doesn't extend to a star-autonomous structure. Morally speaking, the tensor product of would be given by in the forwards direction, and pairs of affine functions satisfying in the backwards direction, with either of these expressions giving the pairing. Since the first formula implies the pairing is convex in (as it must be), but the latter implies it's concave in , must take values only those so that is affine, and similarly for . This can easily be an empty set, but in a star-autonomous category we always have a canonical costate , which would be impossible in that case. Eigil Fjeldgren Rischel 2024 4 23 https://erischel.com/lcc-001K/ lcc-001K /lcc-001K/ Proposition The category of minmax problems has products, given by Here denotes the coproduct of convex spaces, and is a generic element (note that ) By duality (with ), it also has coproducts given by . Backlinks Related Contributions