What are graph neural networks (GNN)?

That Transform Technology Summits launch on October 13 with Low-Code / No Code: Enabling Enterprise Agility. Register now!

Graphs are all around us. Your social network is a graph of people and relationships. It’s your family too. The paths you take to go from point A to point B form a graph. The links that link this web page to others form a graph. When your employer pays you, your payment goes through a graph of financial institutions.

Basically, anything that consists of linked entities can be represented as a graph. Graphs are excellent tools for visualizing relationships between people, objects, and concepts. In addition to visualizing information, however, graphs can also be good data sources for training machine learning models for complex tasks.

Graph Neural Network (GNN) is a type of machine learning algorithm that can extract important information from graphs and make useful predictions. With graphs becoming more pervasive and richer with information, and artificial neural networks becoming more popular and skilled, GNNs have become a powerful tool for many important applications.

Transforming graphs for the treatment of neural networks

Each graph is composed of notes and edges. For example, in a social network, nodes can represent users and their properties (e.g., name, gender, age, city), while edges can represent the relationships between users. A more complex social graph may include other types of nodes, e.g. Cities, sports teams, news outlets, and borders that describe the relationship between users and these nodes.

Unfortunately, the graph structure is not suitable for machine learning. Neural networks expect to receive their data in a consistent format. Multilayer perceptrons expect a fixed number of input functions. Convolutional neural networks expect a grid that represents the different dimensions of the data they process (e.g., width, height, and color channels in images).

Graphs can come in different structures and sizes, which is not consistent with the rectangular arrays that neural networks expect. Graphs also have other properties that make them different from the type of information that classical neural networks are designed for. For example, graphs are “permutation invariant”, meaning that changing the order and position of the node does not make a difference as long as their relations remain the same. In contrast, changing the order of pixels results in a different image and causes the neural network that processes them to behave differently.

To make graphs useful for deep learning algorithms, their data must be transformed into a format that can be processed by a neural network. The type of formatting used to represent graph data may vary depending on the graph type and the intended application, but in general the key is to represent the information as a series of matrices.

A series of images against a grainy, sand-colored background.  The first is the profiles of a number of people associated with notes.  The next are two graphs with a number of people's first names and basic biographical information.

Consider e.g. A graph on social networks. The nodes can be represented as a table of user properties. The node table, where each row contains information about a device (for example, user, customer, bank transaction), is the type of information that you want to provide to a normal neural network.

But graph neural networks can also learn from other information contained in the graph. The edges, the lines connecting the nodes can be represented in the same way, where each row contains the user’s IDs and additional information such as the date of friendship, relationship type, etc. Finally, the general connection of the graph can be represented as an adjacency matrix showing which nodes are connected to each other.

When all of this information is provided to the neural network, it can extract patterns and insights that go beyond the simple information contained in the individual components of the graph.

Graph deposits

Three images set against a blue marble background.  The first: a series of graphs with users' names and personal information.  Second image: bar chart with the title "Graph embedding." Third image: a spreadsheet with users and numbers with the title "Graph deposits."

Graph neural networks can be created like any other neural network using fully connected layers, interleaving layers, aggregation layers, etc. The type and number of layers depends on the type and complexity of the graph data and the desired output.

GNN receives the formatted graph data as input and produces a vector with numeric values ​​representing relevant information about nodes and their relations.

This vector representation is called “graph embedding”. Embeddings are often used in machine learning to transform complex information into a structure that can be differentiated and learned. For example, natural language processing systems use word embedding to create numerical representations of words and their relationships together.

How does GNN create the graph embedding? When the graph data is sent to GNN, the functions of each node are combined with the properties of its neighboring nodes. This is called “message transfer”. If the GNN is composed of more than one layer, subsequent layers repeat the message transmission, collecting data from neighbors to neighbors and aggregating them with the values ​​obtained from the previous layer. For example, in a social network, the first layer in GNN would combine user data with their friends ‘, and the next layer would add data from friends’ friends and so on. Finally, the output layer of the GNN produces the embedding, which is a vector representation of the node’s data and its knowledge of other nodes in the graph.

Interestingly, this process is very similar to how convulatory neural networks extract functions from pixel data. Therefore, a very popular GNN architecture is Graph Convolution Neural Network (GCN), which uses convolution layers to create graph embedments.

Applications of graph neural networks

An image of three separate neural networks against a gray background.

Once you have a neural network that can learn the embedding in a graph, you can use it to perform various tasks.

Here are a few applications for graph neural networks:

Node Classification: One of the powerful uses of GNNs is to add new information to nodes or fill in gaps where information is missing. Say, for example, that you run a social network and you’ve seen a few bot accounts. Now you want to find out if there are other bot accounts in your network. You can train a GNN to classify other users on the social network as “bot” or “not bot” based on how close their graphs are to those of the known bots.

Edge prediction: Another way to use GNNs is to find new edges that can add value to the graph. Back to our social network, a GNN can find users (nodes) that are close to you in embedding space, but who are not your friends yet (i.e. there is no edge that connects you to each other). These users can then be introduced to you as a friend suggestion.

Clusters: GNNs can get new structural information from graphs. For example, in a social network where everyone is somehow related to others (through friends or friends of friends, etc.), GNN can find nodes that form clusters in the embedding space. These clusters can point to groups of users who share similar interests, activities, or other conspicuous characteristics, no matter how close their relationships are. Clustering is one of the most important tools used in machine learning based marketing.

Graph neural networks are very powerful tools. They have already found powerful applications in domains such as route planning, fraud detection, network optimization and drug research. Wherever there is a graph of related devices, GNNs can help get the most out of existing data.

Ben Dickson is a software engineer and founder of TechTalks. He writes about technology, business and politics.

This story originally appeared on Bdtechtalks.com. Copyright 2021


VentureBeat’s mission is to be a digital urban space for technical decision makers to gain knowledge about transformative technology and transactions. Our site provides important information about data technologies and strategies to guide you as you lead your organizations. We invite you to join our community to access:

  • updated information on topics that interest you
  • our newsletters
  • gated thought-leader content and discount access to our valued events, such as Transform 2021: Learn more
  • networking features and more

sign up

Leave a Comment