GraphStorm 0.3: Scalable multi-task graph learning with a user-friendly API

GraphStorm is a low-code enterprise graph machine studying (GML) framework that allows constructing, coaching, and deploying graph ML options on complicated enterprise-scale graphs in days as a substitute of months. With GraphStorm, you’ll be able to construct options that straight take into account the relationships or interplay constructions between billions of entities which are intrinsically embedded in most real-world knowledge, together with fraud detection situations, suggestions, and group detection. testing and search/retrieval points.

In the present day, we launch GraphStorm 0.3, which provides native assist for graph multi-task studying. Particularly, GraphStorm 0.3 enables you to outline a number of coaching targets on totally different nodes and edges inside a single coaching loop. As well as, GraphStorm 0.3 provides a brand new API to customise the GraphStorm pipeline: you now solely want 12 strains of code to implement a customized node classification coaching loop. That can assist you get began with the brand new API, we now have launched two Jupyter Pocket book examples: one for node classification and one other for the hyperlink prediction activity. We additionally printed a complete research of co-training language fashions (LM) and graph neural networks (GNN) for big graphs with wealthy textual options utilizing the Microsoft Tutorial Graph (MAG) dataset within the KDD 2024 paper. This research demonstrates the efficiency and scalability of GraphStorm on text-rich graphs, in addition to finest practices for configuring GML coaching loops for higher efficiency and effectivity.

Native assist for multi-task studying on graphs

Many enterprise purposes have graph knowledge related to a number of duties on totally different nodes and edges. For instance, a retail group desires to carry out fraud detection on sellers and patrons. Scientific publishers need to discover extra related works to quote of their papers and want to decide on the best matters to make their publications simply discoverable. To higher mannequin such purposes, clients requested us to assist multi-task studying on graphs.

GraphStorm 0.3 helps multi-task studying on graphs, with the six most typical duties: node classification, node regression, edge classification, edge regression, hyperlink prediction and node characteristic reconstruction. You’ll be able to specify coaching objectives by way of YAML configuration recordsdata. For instance, a scientific writer might use the next YAML configuration to additionally outline the paper topic classification activity paper Node and hyperlink prediction duties paper-citing-paper Fringe of Scientific Writer Use Instances:

model: 1.0
    gsf:
        primary: # primary settings of the spine GNN mannequin
            ...
        ...
        multi_task_learning:
            - node_classification:         # outline a node classification activity for paper topic prediction.
                target_ntype: "paper"      # the paper nodes are the coaching targets.
                label_field: "label_class" # the node characteristic "label_class" accommodates the coaching labels.
				mask_fields:
                    - "train_mask_class"   # prepare masks is known as as train_mask_class.
                    - "val_mask_class"     # validation masks is known as as val_mask_class.
                    - "test_mask_class"    # check masks is known as as test_mask_class.
                num_classes: 10            # There are whole 10 totally different lessons (topic) to foretell.
                task_weight: 1.0           # The duty weight is 1.0.
                
            - link_prediction:                # outline a hyperlink prediction paper quotation suggestion.
                num_negative_edges: 4         # Pattern 4 detrimental edges for every constructive edge throughout coaching
                num_negative_edges_eval: 100  # Pattern 100 detrimental edges for every constructive edge throughout analysis
                train_negative_sampler: joint # Share the detrimental edges between constructive edges (to speedup coaching)
                train_etype:
                    - "paper,citing,paper"    # The goal edge kind for hyperlink prediction coaching is "paper, citing, paper"
                mask_fields:
                    - "train_mask_lp"         # prepare masks is known as as train_mask_lp.
                    - "val_mask_lp"           # validation masks is known as as val_mask_lp.
                    - "test_mask_lp"          # check masks is known as as test_mask_lp.
                task_weight: 0.5              # The duty weight is 0.5.

For extra particulars on the way to run graph multi-task studying with GraphStorm, see Multi-task Studying in GraphStorm in our documentation.

New API for customizing GraphStorm pipelines and elements

Since GraphStorm’s launch in early 2023, clients have primarily used its command line interface (CLI), which abstracts the complexity of graph ML pipelines, permitting you to shortly construct, prepare, and deploy fashions utilizing frequent recipes. Nevertheless, clients informed us they needed an interface that made it simpler for them to customise GraphStorm’s coaching and inference pipelines to their particular necessities. Based mostly on buyer suggestions on the experimental API we launched in GraphStorm 0.2, GraphStorm 0.3 introduces a refactored graph ML pipeline API. Utilizing the brand new API, you’ll be able to outline a customized node classification coaching pipeline with simply 12 strains of code, as proven within the following instance:

import graphstorm as gs
gs.initialize()

acm_data = gs.dataloading.GSgnnData(part_config='./acm_gs_1p/acm.json')

train_dataloader = gs.dataloading.GSgnnNodeDataLoader(dataset=acm_data, target_idx=acm_data.get_node_train_set(ntypes=['paper']), fanout=[20, 20], batch_size=64)
val_dataloader = gs.dataloading.GSgnnNodeDataLoader(dataset=acm_data, target_idx=acm_data.get_node_val_set(ntypes=['paper']), fanout=[100, 100], batch_size=256, train_task=False)
test_dataloader = gs.dataloading.GSgnnNodeDataLoader(dataset=acm_data, target_idx=acm_data.get_node_test_set(ntypes=['paper']), fanout=[100, 100], batch_size=256, train_task=False)

mannequin = RgcnNCModel(g=acm_data.g, num_hid_layers=2, hid_size=128, num_classes=14)
evaluator = gs.eval.GSgnnClassificationEvaluator(eval_frequency=100)

coach = gs.coach.GSgnnNodePredictionTrainer(mannequin)
coach.setup_evaluator(evaluator)

coach.match(train_dataloader, val_dataloader, test_dataloader, num_epochs=5)

That can assist you get began with the brand new API, we have additionally launched new Jupyter Pocket book examples within the documentation and tutorial pages.

Complete research of LM+GNN on massive photos with wealthy textual content options

Many enterprise purposes have graphics with textual content options. For instance, in a retail search software, procuring log knowledge offers insights into how text-rich product descriptions, search queries, and buyer habits are associated. Fundamental massive language fashions (LLMs) will not be inherently appropriate for modeling such materials as a result of the underlying materials distributions and relationships don’t correspond to what the LLM has discovered from the pre-trained materials corpus. GML, then again, could be very appropriate for modeling associated supplies (graphs), however till now, GML practitioners needed to manually mix their GML fashions with LLM to mannequin literal options and get the very best outcomes for his or her use circumstances. Finest efficiency. Particularly when the underlying graphics dataset is massive, this handbook effort is difficult and time-consuming.

In GraphStorm 0.2, GraphStorm introduces built-in expertise to effectively prepare language fashions (LM) and GNN fashions at scale on massive quantities of text-rich graphs. Since then, clients have been asking us the way to use GraphStorm’s LM+GNN expertise to optimize efficiency. To handle this subject, we launched a LM+GNN benchmark in GraphStorm 0.3 that makes use of the big graph dataset Microsoft Academy Graph (MAG) and targets two normal graph ML duties: node classification and hyperlink prediction. Graph datasets are heterogeneous graphs containing a whole lot of thousands and thousands of nodes and billions of edges, most of which have wealthy textual options. The detailed statistics of the information set are proven within the desk beneath.

We benchmarked two important LM-GNN strategies in GraphStorm: pre-trained BERT+GNN (a extensively adopted baseline technique) and fine-tuned BERT+GNN (launched by GraphStorm builders in 2022). Utilizing the pre-trained BERT+GNN technique, we first use the pre-trained BERT mannequin to calculate the embedding of node textual content options, after which prepare the GNN mannequin for prediction. By fine-tuning the BERT+GNN technique, we first fine-tune the BERT mannequin on graph knowledge, and use the ensuing fine-tuned BERT mannequin to calculate embeddings, that are then used to coach the GNN mannequin for prediction. Relying on the duty kind, GraphStorm offers totally different strategies to fine-tune the BERT mannequin. For node classification, we use the node classification activity to fine-tune the BERT mannequin on the coaching set; for hyperlink prediction, we use the hyperlink prediction activity to fine-tune the BERT mannequin. Within the experiment, we used 8 r5.24xlarge cases for knowledge processing and 4 g5.48xlarge cases for mannequin coaching and inference. In contrast with the pre-trained BERT+GNN, the efficiency (hyperlink prediction on MAG) of the fine-tuned BERT+GNN technique is improved by 40%.

The desk beneath reveals the mannequin efficiency of each strategies and the general computation time of your complete pipeline ranging from knowledge processing and graph development. NC represents node classification, and LP represents hyperlink prediction. LM Time Value refers back to the time spent calculating the BERT embedding and the time spent fine-tuning the BERT mannequin of pre-trained BERT+GNN and fine-tuned BERT+GNN respectively.

We additionally benchmark GraphStorm on massive artificial graphs to show its scalability. We generate three artificial graphs with 1 billion, 10 billion and 100 billion edges. The corresponding coaching set sizes are 8 million, 80 million, and 800 million respectively. The next desk reveals the computation time for graph preprocessing, graph partitioning, and mannequin coaching. General, GraphStorm can obtain graph development and mannequin coaching for 100 billion-scale graphs in a matter of hours!

Extra benchmark particulars and outcomes are supplied in our KDD 2024 paper.

in conclusion

Launched underneath the Apache-2.0 license and designed that will help you sort out large-scale graph ML challenges, GraphStorm 0.3 now provides native assist for multi-task studying and new APIs for customizing GraphStorm pipelines and different elements. See the GraphStorm GitHub repository and documentation to get began.

In regards to the creator

Music Xiang is a Senior Software Scientist at AWS AI Analysis and Training (AIRE), the place he developed deep studying frameworks together with GraphStorm, DGL, and DGL-KE. He led the event of Amazon Neptune ML, a brand new characteristic of Neptune that makes use of graph neural networks to retailer graphs in a graph database. He now leads the event of GraphStorm, an open supply graph machine studying framework for enterprise use circumstances. He acquired his Ph.D. In 2014, he obtained a PhD in pc programs and structure from Fudan College in Shanghai.

Zhang Jian is a senior software scientist who has been utilizing machine studying expertise to assist clients clear up varied issues, reminiscent of fraud detection, ornamental picture era, and many others. He has efficiently developed graph-based machine studying, particularly graph neural community options, for purchasers in China, america, and Singapore. As an enlightener of AWS graphics capabilities, Mr. Zhang has given many public lectures on GNN, Depth Graphics Library (DGL), Amazon Neptune and different AWS providers.

Florian Thorpe He’s the Principal Technical Product Supervisor for AWS AI/ML Analysis, supporting scientific groups such because the Graph Machine Studying group and ML programs groups engaged on large-scale distributed coaching, inference, and failure restoration. Previous to becoming a member of AWS, Florian led product administration for autonomous driving applied sciences at Bosch, served as a method marketing consultant at McKinsey & Firm, and as a management programs/robotics scientist (he holds a PhD on this discipline).