Topic 6: What is KAN?

We live in a time of change and revision as we need better algorithms, more powerful computing, and new levels of AI. Efficiency is a new trending term in the AI world that has come to replace previous battles for the biggest models. With that, some fundamental approaches are being reconsidered.

For example, the multilayer perceptron (MLP), arguably, the most important algorithm in the history of deep learning received an alternative just recently. A group of researchers proposed Kolmogorov–Arnold Networks (KAN) reporting better accuracy and interpretability than MLP in certain tasks.

What is KAN? How can it improve the results achieved by the fundamental MLP? Let’s find out.

In today’s episode, we will cover:

• First, what is MLP and where it came from?

• KAN story

• KAN architecture

• Is KAN an improved MLP?

• Advantages of KANs over MLPs

• KAN’s limitations

• Conclusion

• Bonus: Resources

First, what is MLP and where it came from?

Multilayer perceptrons (MLPs), a core type of feedforward neural network, are fundamental in artificial intelligence. Among different types of neural networks (NNs)*, feedforward neural networks (FNNs) are the simplest. Information flows only in one direction, from input to output, without loops or cycles in the network architecture.

*Neural networks are inspired by the processes happening in the human brain, where biological neurons work together to identify phenomena, weigh options, and arrive at conclusions.

To understand MLPs, let’s revise some basics of neural networks. Multilayer perceptron consists of layers of nodes (also known as neurons or perceptrons):

• Input Layer: This is the initial point of data entry into the network. Each node here represents a different feature of the input data, effectively translating raw data into a format the network can work with.

• Hidden Layers: Situated between the input and output layers, these layers can vary in number and size depending on the network's complexity. Each neuron in these layers processes inputs from all the neurons in the previous layer, transforming them via weights, biases, and activation functions, then passing the result to the next layer.

• Output Layer: This final layer outputs the network's predictions or classifications. The number of neurons here aligns with the desired output dimensions, depending on the specific task at hand.

Other important concepts: