Category Theory
Zulip Server
Archive

You're reading the public-facing archive of the Category Theory Zulip server.
To join the server you need an invite. Anybody can get an invite by contacting Matteo Capucci at name dot surname at gmail dot com.
For all things related to this archive refer to the same person.

Stream: event: Categorical Probability and Statistics 2020 workshop

Topic: Jun 6: Evan Patterson's talk

Paolo Perrone (Jun 04 2020 at 19:13):

Hey all,
This is the discussion thread of Evan Patterson's talk, "The algebra of statistical theories and models".
The talk, besides being on Zoom, is livestreamed here: https://youtu.be/kKDMCDUaxxE

Paolo Perrone (Jun 04 2020 at 19:23):

Date and time: Saturday, 6 Jun, 16h UTC.

Tobias Fritz (Jun 06 2020 at 15:50):

The talk starts in 10 mins!

Tomáš Gonda (Jun 06 2020 at 16:36):

An open-ended philosophy question: Is the "obliteration of the distinction between syntax and semantics" a good thing for the development of science going forward? By the way, it might be helpful if @Evan Patterson specified what he meant by this, I am not quite sure to be honest.

More concretely, and related to today's talks, does it (what taken at face value) pose a barrier to the unscrambling of the omelette of influence and inference in statistics (which, of course, far predates Suppes)? It is interesting to me (and kind of paradoxical in relation to the above quote) that a natural way for @Rob Spekkens to proceed in this unscrambling (following Jaynes and Pearl) has been in the direction of categorical logic and functorial semantics.

Nathanael Arkor (Jun 06 2020 at 16:53):

For classical algebraic theories, there are two flavours of model: cartesian functors into Set and cartesian functors into arbitrary cartesian categories. Set is canonical in some sense, because models in Set have particularly nice properties (e.g. a correspondence with monad algebras). In your setting, you only consider models in Stat. Is this because Stat is canonical in a similar way that Set is (for instance, are statistical theories Stat-enriched?), or because it is a setting that is particularly interesting for your applications (implying that it would be reasonable to take models in other categories with sufficient structure)?

Tomáš Gonda (Jun 06 2020 at 17:09):

What are PROPs, by the way? I am not familiar with the concept.

Arthur Parzygnat (Jun 06 2020 at 17:09):

Here's one of the papers Lawvere mentioned at the end: Xiao-Qing Meng "Categories of Convex Sets and of Metric Spaces." (It's a thesis) I'm curious if we could find a copy somehow?

Nathanael Arkor (Jun 06 2020 at 17:11):

Tomáš Gonda said:

What are PROPs, by the way? I am not familiar with the concept.

They're to algebraic theories what symmetric monoidal categories are to cartesian categories. They're also called one-sorted symmetric monoidal theories.

Tobias Fritz (Jun 06 2020 at 17:12):

Arthur Parzygnat said:

Here's one of the papers Lawvere mentioned at the end: Xiao-Qing Meng "Categories of Convex Sets and of Metric Spaces." (It's a thesis)

Thanks! It seems that the nLab has a copy of Meng's thesis.

Tomáš Gonda (Jun 06 2020 at 17:13):

That was a very interesting and thought-provoking talk, @Evan Patterson, thanks! I will be sure to check out your thesis :slight_smile:

Arthur Parzygnat (Jun 06 2020 at 17:23):

What is Kolmogorov consistency? From what I could tell while looking at http://www.its.caltech.edu/~kcborder/Notes/Kolmogorov.pdf, it seems that the data of this definition consists of a fixed set $X$ together with a net $(\Sigma_{\alpha},p_{\alpha})$ of $\sigma$-algebras on $X$ and probability measures $p_{\alpha}$ on $(X,\Sigma_{\alpha})$ ordered by inclusion, namely $\Sigma_{\alpha}\subseteq\Sigma_{\beta}$ and $(p_{\beta})_{\restriction\alpha}$ whenever $\alpha\le\beta$. These data can be described as a functor from the ordered set of $\alpha$'s to morphisms of probability spaces $(X,\Sigma_{\alpha},p_{\alpha})\xrightarrow{\mathrm{id}_{X}}(X,\Sigma_{\beta},p_{\beta})$. The Kolmogorov extension property seems to say that some colimit exists? So was the question in the zoom discussion a question regarding whether certain axioms of a Markov category guarantee that such colimits exist?

Evan Patterson (Jun 06 2020 at 17:25):

Tomáš Gonda said:

An open-ended philosophy question: Is the "obliteration of the distinction between syntax and semantics" a good thing for the development of science going forward? By the way, it might be helpful if Evan Patterson specified what he meant by this, I am not quite sure to be honest.

OK, I got a bit carried away :) I should say this more mildly. In traditional logic, there is a sharp boundary between syntax, which is formal, symbolic language, and semantics, which is usually ordinary mathematical objects. In categorical logic, this distinction is much less sharp because theories are algebraic structures. Thus, they exist independent of any particular syntax or presentation. Just as a group exists independent of its presentation, so does the theory of groups (as a Lawvere theory) exist independent of its presentation. Note the level shift! That being said, you still typically specify theories by presenting them, so syntax doesn't actually go away. And of course, if you want to specify theories on a computer, formal syntax is essential.

More concretely, and related to today's talks, does it (what taken at face value) pose a barrier to the unscrambling of the omelette of influence and inference in statistics (which, of course, far predates Suppes)? It is interesting to me (and kind of paradoxical in relation to the above quote) that a natural way for Rob Spekkens to proceed in this unscrambling (following Jaynes and Pearl) has been in the direction of categorical logic and functorial semantics.

I can't say much about this now (Rob's talk was very early in the morning for me!), but I look forward to catching up on this interesting work.

Evan Patterson (Jun 06 2020 at 17:27):

Tomáš Gonda said:

That was a very interesting and thought-provoking talk, Evan Patterson, thanks! I will be sure to check out your thesis :)

Thank you!

Evan Patterson (Jun 06 2020 at 17:36):

Arthur Parzygnat said:

What is Kolmogorov consistency? From what I could tell while looking at http://www.its.caltech.edu/~kcborder/Notes/Kolmogorov.pdf, it seems that the data of this definition consists of a fixed set $X$ together with a net $(\Sigma_{\alpha},p_{\alpha})$ of $\sigma$-algebras on $X$ and probability measures $p_{\alpha}$ on $(X,\Sigma_{\alpha})$ ordered by inclusion, namely $\Sigma_{\alpha}\subseteq\Sigma_{\beta}$ and $(p_{\beta})_{\restriction\alpha}$ whenever $\alpha\le\beta$. These data can be described as a functor from the ordered set of $\alpha$'s to morphisms of probability spaces $(X,\Sigma_{\alpha},p_{\alpha})\xrightarrow{\mathrm{id}_{X}}(X,\Sigma_{\beta},p_{\beta})$. The Kolmogorov extension property seems to say that some colimit exists? So was the question in the zoom discussion a question regarding whether certain axioms of a Markov category guarantee that such colimits exist?

I thought the question during the talk was about consistency of estimators, but now I'm thinking that I misinterpreted the question, since Kolmogorov's name doesn't seem to be attached to that kind of consistency. If the question is about how the Kolmogorov extension theorem, aka the "Kolmogorov consistency theorem," is related to Markov categories, then I don't know.

Evan Patterson (Jun 06 2020 at 17:50):

Nathanael Arkor said:

For classical algebraic theories, there are two flavours of model: cartesian functors into Set and cartesian functors into arbitrary cartesian categories. Set is canonical in some sense, because models in Set have particularly nice properties (e.g. a correspondence with monad algebras). In your setting, you only consider models in Stat. Is this because Stat is canonical in a similar way that Set is (for instance, are statistical theories Stat-enriched?), or because it is a setting that is particularly interesting for your applications (implying that it would be reasonable to take models in other categories with sufficient structure)?

That's an interesting question. I never thought about it. I just had the naive idea that Stat was a convenient setting. It would be nice to know whether it or some other category is canonical in some sense.

Bradley Saul (Jun 06 2020 at 18:56):

@Evan Patterson Great talk. Speaking as a statistician, I'm quite excited about this. Have you started thinking about software to implement these ideas?

Evan Patterson (Jun 06 2020 at 19:55):

Thanks! I've thought about it but I haven't yet got further than that :) I would probably start by implementing statistical theories in Julia, building on Catlab.jl, and then interface with one or more existing probabilistic programming languages, maybe Turing.jl or Stan via Stan.jl. For me, the dream is to build a system where statistical theories and models can be treated like data, a computable resource that can be inspected and manipulated. I believe that the category theory and categorical logic provide the right mathematical toolbox, but making it all work as a practical software system is a big job. If anyone is interested in working with me on that, please let me know!

Bradley Saul (Jun 06 2020 at 20:12):

I would love to participate. To me, this is right direction for statistical computing. I think we need to have a strong theoretical background, rather than a hodge-podge of routines. I'd like to see unified statistical modeling systems built "from the math up", rather than from interfaces down (not to take away from the utility of R packages like parsnip.

I know you've put a lot of work into Catlab, but have you thought about a statically typed language like Haskell or perhaps even a dependently typed language like Idris? I've been programming a bit in Haskell for ~1yr, and thusfar, I've found that a pure functional language with static types both (a) makes my life easier as a developer and (b) gives me more confidence that I correctly implemented whatever mathematical constructs I'm working on in my software.

Evan Patterson (Jun 06 2020 at 20:27):

That's great! We should talk more.

The question of what language to build in comes up fairly often. For now, I am committed to working in Julia; in fact, I will soon be ramping up my efforts on Catlab as part of other projects. I think Julia is a good compromise language because it is connected to the numerical and scientific computing communities in a way that has Haskell has never managed to be, but compared to Python or R, Julia has a nicer type system and other language features that make it better suited to the algebraic approach to computing. Also, certain features of Catlab are related to/inspired by things from Haskell. I am thinking especially of its system for generalized algebraic theories (GATs), which are reminiscent of Haskell's type classes. But of course I understand that everyone has their own views about such things.

Bradley Saul (Jun 06 2020 at 21:09):

Evan Patterson said:

That's great! We should talk more.

:thumbs_up:

The question of what language to build in comes up fairly often. For now, I am committed to working in Julia; in fact, I will soon be ramping up my efforts on Catlab as part of other projects. I think Julia is a good compromise language because it is connected to the numerical and scientific computing communities in a way that has Haskell has never managed to be, but compared to Python or R, Julia has a nicer type system and other language features that make it better suited to the algebraic approach to computing.

Sounds good to me. I don't have a lot of familiarity with Julia, but I've heard great things about it.

Paolo Perrone (Jun 07 2020 at 02:31):

Hi all! Here's the video.
https://youtu.be/BhKaHAY8Ec8

Arthur Parzygnat (Jun 07 2020 at 06:38):

At the end of Evan's talk, Lawvere mentioned Peter J. Huber. Here is an interview with Peter https://projecteuclid.org/euclid.ss/1215441288 (arxiv link: https://arxiv.org/abs/0808.0777). Haven't read it in full yet, but it describes how and why he transitioned from category theory to statistics. You can also find some of his earlier papers on category theory (here's one https://link.springer.com/article/10.1007/BF01441134). As far as I know, he's not mentioned in the nlab.

Faez Shakil (Jun 07 2020 at 08:16):

@Evan Patterson when you say a computable resource they can be inspected and manipulated are you referring to composable density representations and inference procedures or do you have a broader intuition in mind?

Evan Patterson (Jun 07 2020 at 18:24):

Arthur Parzygnat said:

At the end of Evan's talk, Lawvere mentioned Peter J. Huber. Here is an interview with Peter https://projecteuclid.org/euclid.ss/1215441288 (arxiv link: https://arxiv.org/abs/0808.0777). Haven't read it in full yet, but it describes how and why he transitioned from category theory to statistics. You can also find some of his earlier papers on category theory (here's one https://link.springer.com/article/10.1007/BF01441134). As far as I know, he's not mentioned in the nlab.

Thanks, that's really interesting. I didn't know Huber began his career working in category theory. In statistics, he is best known for pioneering work on robust statistics: estimators that are robust to some fraction of corruption or outliers in the data. I've always thought that robustness is an underrated topic in statistics.

Evan Patterson (Jun 07 2020 at 18:44):

Faez Shakil said:

Evan Patterson when you say a computable resource they can be inspected and manipulated are you referring to composable density representations and inference procedures or do you have a broader intuition in mind?

I should have elaborated, as I had something a bit different in mind. I was thinking about treating statistical models as logical/algebraic structures, ultimately leading to useful data structures on a computer, so that different models applicable to a single study, or models taken from related studies, can be compared in a systematic way. Once you have a good computational representation, you can start to think about more intelligent data science tools, that would help people explore the space of possible models or conduct meta-analysis or other metascientific tasks. Again, that's a really big project but I think it's a nice vision to work towards. If you really want to hear me wax poetic about such things, you might take a look at the Introduction (Ch. 1) of my thesis.