You're reading the public-facing archive of the Category Theory Zulip server.
To join the server you need an invite. Anybody can get an invite by contacting Matteo Capucci at name dot surname at gmail dot com.
For all things related to this archive refer to the same person.
Harold Kimmey (Apr 18 2024 at 23:05):I found this article and was wondering what the ACT community thought about it: "Unifying Neural Network Design with Category Theory: A Comprehensive Framework for Deep Learning Architecture" By Sana Hassan
"They have proposed a solution grounded in category theory, aiming to create a more integrated and coherent methodology for neural network design. This innovative approach encapsulates the diverse landscape of neural network designs, including recurrent neural networks (RNNs), and offers a new lens to understand and develop deep learning architectures. By applying category theory, the research captures the constraints used in Geometric Deep Learning (GDL) and extends beyond to a wider array of neural network architectures."
Kevin Carlson (Arlin) (Apr 18 2024 at 23:37):I'd try looking at the original paper and/or asking a more specific question.
Peva Blanchard (Apr 19 2024 at 05:41):One of the authors of the cited paper is Bruno Gavranovic. He presents his work in the "community: our work" thread. There is also a start-up, symbolica which, I think, aims at providing categorical tools for nn architecture, as explained in the paper.
As you can read in the zulip stream above, there is a concern here that the ACT community may be hit by the AI hype.
Harold Kimmey (Apr 19 2024 at 13:03):First, my apologies as I a fairly new to Category Theory and not a Mathematician. My background is in Computer Science and Business applications of technology, primarily for profit (Return on Investment/ROI). My experience is fairly broad and deep, implementing technologies earlier than most by finding how they could connect to profitability earlier. I "think" there are profitable business application for Category Theory and was trying to start a conversation with the purpose of learning more about Category Theory while trying to connect the dots to ROI.
While ROI tends to distract from some research in a technology it often brings funding that otherwise would never happen.
Regarding the paper, maybe @Bruno Gavranović would help the discussion.
Do you think that the application of Category Theory would help enhance the "exploitability" of Neural Networks, or of AI in general?
John Baez (Apr 19 2024 at 14:41):I think category theory should be very helpful, but it's important to keep in mind that category theory is a very general, abstract set of ideas and principles. So, to apply it to any particular problem you have to either hope that someone has already successfully applied it to something quite similar, or be prepared to put in a lot of work. In the latter case you need a team including experts on the problem and experts on category theory, and they need to talk to each other a lot.
I know this because I've been working for about 14 years to apply category theory to modeling, and though I managed to make a lot of progress laying the foundations with some grad students and other mathematicians, the project really took off around 4 years ago when I teamed up with a group of epidemiologists and computer scientists, most of whom know quite a bit of category theory. We are now writing software for epidemiological modeling based on category theory, and it's quite exciting.
John Baez (Apr 19 2024 at 14:52):The paper you point to was written by a team of people working for a company, Symbolica, that has now gotten about $30 million in startup funds to develop a new approach to machine learning. Much of the paper is an introduction to category theory. It's not the case that anyone can just read that paper, take the time to understand it in detail, and then rapidly start applying category theory to machine learning. The paper is not a recipe for machine learning: it's a sketch of a plan. So I expect this team will need to work at least a year or two, and maybe longer, before getting commercially applicable results. Luckily some of these people know a lot of category theory. I believe others are experts on machine learning. Anyway, that kind of mix is what's required to make progress.
John Baez (Apr 19 2024 at 14:57):If you want to fund commercializable applications of category theory I suggest talking not only to the authors of that paper but also Shaowei Lin and Brendan Fong at the Topos Institute. This is an institute in Berkeley devoted to working on applications of category theory. I'm doing my work on epidemiological modeling with some people there, and I have a very high opinion of this institute.
Harold Kimmey (Apr 19 2024 at 16:00):Thank you John for your thoughts and the connection to Shaowei Lin and Brendan Fong at the Topos Institute!
I had recently heard of Symbolica. I was only familiar with George Morgan, who had been at Tesla. A friend, @Brandon Baylor, had mentioned an article about Symbolica recently, but was not aware of the connection between the paper and the article.
It would seem that using Category Theory would be a good fit for modeling the spread of diseases within populations due to the categories of populations and relationships between those populations (I may have very badly misstated this here, but trying...).
John Baez (Apr 19 2024 at 17:41):If you want to read a bit more about categories in epidemiology, or watch a talk about it, you can go here. I know it's not what you're mainly interested in, but it's an example of applied category theory that's reaching the point of practical applications, and I focus on the challenges of building a working team.
Another good example is the work that people are doing at the company Quantinuum, which does quantum computing, but also work related to AI. (I've linked to something about AI.)
And another is the work @Ryan Wisnesky and others are doing at Conexus, which does novel things with databases.
Dylan Braithwaite (Apr 21 2024 at 10:05):John Baez said:
The paper you point to was written by a team of people working for a company, Symbolica.
For what it's worth, I think only some of the authors are associated with Symbolica - others are from Google DeepMind.
I say this because it's maybe interesting that a big established organisation like DeepMind is taking interest in category theory for machine learning too.
John Baez (Apr 21 2024 at 10:32):Thanks for clarifying that!
Harold Kimmey (Apr 22 2024 at 03:26):John Baez said:
If you want to read a bit more about categories in epidemiology, or watch a talk about it, you can go here.
I watched the talk and found it a little easier to understand now.
John Baez (Apr 22 2024 at 12:26):Great!
John Baez (Apr 22 2024 at 12:27):There's more to it all than what I explained in that talk, but that's the key idea.