Category Theory
Zulip Server
Archive

You're reading the public-facing archive of the Category Theory Zulip server.
To join the server you need an invite. Anybody can get an invite by contacting Matteo Capucci at name dot surname at gmail dot com.
For all things related to this archive refer to the same person.

Stream: learning: questions

Topic: Distributions over objects of a category

David Corfield (May 15 2025 at 09:37):

I feel this must be well-known, but can't see it explicitly anywhere. Is there a standard way to generate out of a category, $\mathcal{C}$, some kind of category of probability distributions over $Ob(\mathcal{C})$? Or perhaps a category of $\mathbb{R}^+$-weighted sums of objects of $\mathcal{C}$.

John Baez (May 15 2025 at 10:10):

If you want isomorphic objects to have the same probability, you might instead want probability distributions on the set of isomorphism classes.

John Baez (May 15 2025 at 10:12):

One interesting thing is that any groupoid of finite cardinality has a canonical probability distribution on its set of isomorphism classes. And in many cases you can make this temperature-dependent and get a 1-parameter family.

David Corfield (May 15 2025 at 10:27):

Thanks. I can see decategorifying a groupoid to a set provides an answer. But I'm wondering about some generalization of the Families as free coproduct completion construction to allow weightings.

Don't the probabilistic programming people meet this issue where the result of a random sample determines which object appears as the codomain of the computation?

David Corfield (May 15 2025 at 11:10):

Or do people look on the latter as a probabilistic map into the coproduct of possible target objects?

James Deikun (May 15 2025 at 11:13):

Probably the only probabilistic programming semantics that has dependent types is Bart Jacobs' topos-based one.

James Deikun (May 15 2025 at 11:15):

Sorry. maybe it's Jules Jacobs?

James Deikun (May 15 2025 at 11:16):

Nope, Bart it was.

James Deikun (May 15 2025 at 11:26):

But that type theory doesn't have dependent types either, it uses coproducts as you speculated.

Tobias Fritz (May 15 2025 at 11:34):

And even if you have a notion of sampling from a dependent type $B : A \to \mathsf{Type}$, I'd expect that the programmer would have to a specify a distribution on $A$ first, together with a family of distributions over the $B(a)$. So I'd say that there doesn't need to be any canonical choice of a distribution on $A$.

James Deikun (May 15 2025 at 11:37):

The ask wasn't a canonical choice of a distribution on $A$, rather it was a sensible notion of distribution on $\mathsf{Type}$.

Tobias Fritz (May 15 2025 at 11:43):

If that's the question, then why not have a type former $P$ that corresponds to the Giry monad? The notion of distribution on $\mathsf{Type}$ is then given by $P(\mathsf{Type})$. For the most naive kind of semantics, one could perhaps just consider $\mathsf{Set}$ with universes, and take $P$ to be the discrete distribution monad.

James Deikun (May 15 2025 at 11:46):

Yes, there are ways to do it if you consider the universe as a mere set, but the idea is that a sensible semantics of distributions on the universe considers that the universe is also a category and constructs a category of distributions! The interesting question here is "what are the arrows?"

John Baez (May 15 2025 at 11:50):

Returning to David's original question: I'm not familiar of a standard way to define a category of probability distributions on the set of objects of a category $\mathsf{C}$. Of course the concept of a probability distribution on the set of objects of $\mathsf{C}$. is well-defined, as is the (equivalence-invariant :thumbs_up:) concept of a probability distribution on the set of isomorphism classes of objects of $\mathsf{C}$. To get a category of these, we just need to choose a concept of morphism between them. The question is what information we want these morphisms to convey?

For example the set of probability distributions on any set has a preorder called the majorization order, which is very interesting in information theory: for many concepts of entropy, if one probability distribution majorizes another it has less entropy. So that's one thing we could use, if we're happy with a preorder.

Tobias Fritz (May 15 2025 at 11:54):

James Deikun said:

Yes, there are ways to do it if you consider the universe as a mere set, but the idea is that a sensible semantics of distributions on the universe considers that the universe is also a category and constructs a category of distributions! The interesting question here is "what are the arrows?"

That sounds like a different question yet again, but one possible answer might be to take the nerve of the category, apply the distribution functor to that, and then consider the resulting new simplicial set as the "category" that you're looking for. This won't be technically a category because it will fail the Segal condition, but it can plausibly be used for some of the same purposes. But perhaps it depends on what you want to do with it; I don't yet see the relevance for probabilistic programming with dependent types.

James Deikun (May 15 2025 at 12:03):

One might define $\mathrm{Hom}(X,Y)$ as a distribution on the arrows out of each object of $C$ so that, if you sample from $X$ and then from the arrows out of the resulting object, the resulting distribution on the codomain is $Y$.

James Deikun (May 15 2025 at 12:06):

(This category won't generally be even locally small, although I think it will become locally small if you limit the distributions on arrows to only come out of the support of $X$.)

James Deikun (May 15 2025 at 12:09):

This construction can become more interesting if you deal in categories internal to measurable spaces so you can have continuous distributions.

David Corfield (May 15 2025 at 12:28):

Thanks for all of your thoughts. Useful to see that things here are not straightforwardly known. I probably need to think harder about what it is I'm after.

Also, I'd forgotten that I meant to take a look at Toby Smithe's Copy-composition for probabilistic graphical models, in particular, prompted by the discussion, to check out his claim

we introduce a useful bifibration of measure kernels, to provide semantics for the notion of stochastic term, which allows us to generalize probabilistic modelling from product to dependent types.