AI-Based Patent Classification: Key NLP Frameworks

AI is transforming patent classification, making the process faster, more accurate, and scalable. Traditionally reliant on manual efforts, this field now leverages NLP-powered models to handle the complexity of assigning classification codes like IPC and CPC. Here's what you need to know:

• Why AI Matters: Over 3 million patents are filed annually, each requiring classification into one of 70,000 IPC subclasses. Manual methods are slow, inconsistent, and costly.

• Key Tools: Models like BERT, PatentSBERTa, and newer LLMs (e.g., LLaMA-3, Qwen) outperform older methods in accuracy, especially for rare or emerging technologies.

• Datasets: USPTO-70k and USPTO-3M are widely used benchmarks, with text fields like Title + Abstract + Claims (TAC) offering the best results.

• Metrics: Top-k accuracy and hierarchical F1 scores are critical for evaluating models, especially given the multilabel nature of patents.

• Efficiency vs. Flexibility: Encoder-based models like PatentSBERTa excel in frequent subclasses due to their energy efficiency, while LLMs are better for rare categories.

For professionals, the choice of model depends on the use case: encoders for high-volume tasks, LLMs for niche classifications, or hybrid approaches for mixed needs. Future trends include multimodal models integrating text and visuals, and continual learning for evolving taxonomies.

Datasets and Evaluation Metrics in Patent Classification Research

Key Patent Datasets

The success of any AI classification model hinges on the quality of the data it's trained and tested with. In the world of patent classification, a few datasets have emerged as benchmarks for researchers using top patent tools.

Among these, the USPTO-70k dataset is particularly suited for practical testing scenarios. It uses a chronological split, training on patents filed from 2006 to 2017, validating with 2018 data, and testing on 2019 filings. This setup closely mimics how real-world models encounter new patent submissions. Meanwhile, USPTO-3M is ideal for training high-capacity models like PatentBERT, which require extensive labeled data.

When it comes to text fields, selecting the right combination is crucial. While claims alone can deliver accurate classifications, many researchers prefer using Title + Abstract + Claims (TAC) for better results. However, TAC can present challenges, as claims often exceed 1,200 tokens, creating memory constraints. These dataset-specific considerations directly influence how performance metrics are chosen for patent classification tasks.

Evaluation Metrics and Known Challenges

Given the complexity of patent datasets, selecting the right evaluation metrics is just as important as the data itself. Standard metrics like precision, recall, and F1 score are commonly used, but patent classification introduces unique challenges due to its multilabel nature. On average, each patent carries 4.16 IPC labels, with some patents exceeding 100 labels. This makes top-k accuracy particularly useful, as it checks whether the correct label appears within the model's top predictions.

One major hurdle lies in choosing the most appropriate F1 score. Micro-averaged F1 often looks impressive since it gives more weight to frequent CPC subclasses. However, macro-averaged F1 paints a more balanced picture by treating all classes equally, highlighting a model's struggles with rare or emerging technology categories.

"Encoder-based models achieve higher aggregate performance... but struggle on rare [subclasses]. In contrast, LLMs achieve relatively higher performance on infrequent subclasses." - Lorenzo Emer et al.

This distinction matters because the IPC system includes around 70,000 subclasses, and the distribution is heavily skewed. A small number of codes dominate most patents, while thousands of niche subclasses appear infrequently. Models optimized for micro-averaged scores may perform well overall but fail to account for this "long tail" of rare technologies.

To address this, hierarchical F1 offers another layer of insight. By considering the nested structure of IPC/CPC classifications (Section > Class > Subclass > Group), it penalizes errors that propagate down the hierarchy. Reporting metrics like micro, macro, and hierarchical F1 together provides a more comprehensive view of a model's real-world performance.

Classical ML and Feature-Based NLP Methods for Patent Classification

Feature Engineering and Classical Algorithms

Before 2016, patent classification primarily relied on traditional information retrieval (IR) techniques and classical machine learning (ML) methods. These methods transformed raw patent texts - titles, abstracts, and claims - into sparse, high-dimensional vectors using the Bag-of-Words (BoW) model. Among these, TF-IDF (Term Frequency–Inverse Document Frequency) was the most widely used representation scheme.

Supervised algorithms like Support Vector Machines (SVMs) and k-Nearest Neighbors (KNNs) were then deployed to predict patent labels corresponding to IPC or CPC codes. To improve performance, researchers invested heavily in feature engineering, incorporating elements like n-grams, skipgrams (allowing up to two skips), and PoS (Part-of-Speech) filtering. As Eva D'hondt et al. highlighted:

"Noun-noun combinations and – to a lesser extent – the adjectival-noun combinations are the most informative phrasal features for patent classification."

These approaches were straightforward and scalable, becoming the standard benchmarks for research. Notably, they were central to the CLEF-IP competitions in 2010 and 2011, which utilized patent datasets spanning filings from 1978 to 2009. These classical pipelines set the stage for the neural methods that followed.

Performance and Use Cases

Classical models performed well for broad classification tasks, such as sorting patents into general IPC sections or classes, encompassing roughly 120 categories. However, their precision declined as classification tasks became more granular. For example, one state-of-the-art classical model achieved 83.50% precision but required 4,000 characters from the description field and extensive manual feature engineering to do so.

Despite these successes, limitations remained. As Lorenzo Emer et al. pointed out:

"BoW-based methods are fundamentally limited by their inability to capture semantic similarity beyond lexical overlap and by their sensitivity to vocabulary drift in rapidly evolving technological domains."

In cases where patents described the same invention using different terminology, classical models struggled to recognize semantic similarities. While these models had lower accuracy in rare subclasses, they offered significant advantages in terms of efficiency. Classical and encoder-based models could be up to three orders of magnitude faster and more energy-efficient during inference, making them essential for large-scale classification pipelines.

Neural NLP Frameworks for Patent Classification

Embedding Techniques and Neural Architectures

The move from classical methods to neural NLP reshaped how patent text is represented. Neural models introduced dense, low-dimensional embeddings that capture semantic meaning rather than relying on word frequency.

This shift paved the way for early neural approaches to tackle the shortcomings of older methods. Static word embeddings like Word2Vec and FastText transformed technical terms into continuous vector spaces. For instance, these models could recognize that "semiconductor substrate" and "silicon wafer" describe related concepts. These embeddings were then processed by deeper architectures: Convolutional Neural Networks (CNNs) to detect local n-gram patterns in claims and abstracts, and LSTMs or GRUs to handle sequential dependencies in longer text spans.

"CNN-based models demonstrated improvements over BoW+SVM pipelines by capturing local n-gram patterns in patent text." - Lorenzo Emer, Researcher, Scuola Superiore Sant'Anna

One major hurdle was the length of patent documents. Patent claims often average 1,200 tokens and can sometimes exceed 10,000. To address this, researchers applied noun extraction techniques using morphological analysis. This reduced token counts by 98.6%, lowering memory demands to just 1.4% while preserving the technical essence of the text.

Research Findings

Neural models consistently surpassed classical pipelines, particularly in multi-label and fine-grained classification tasks. While SVMs often struggled with patents that spanned multiple technological fields, CNN and LSTM architectures proved better at managing overlapping label assignments.

Static embeddings like Word2Vec and FastText, though effective at capturing semantic similarity, fell short in distinguishing between different contextual meanings - a critical issue in patent language. For example, the word "field" could refer to an agricultural area or an electromagnetic field depending on the context. Contextual embeddings from Transformer-based models addressed this limitation by adapting each word’s representation based on its surrounding text. This innovation marked a significant leap forward in patent classification.

Another key insight from this research phase is the importance of combining text fields. Studies showed that using Title + Abstract + Claims (TAC) as input consistently outperformed relying on any single field. Notably, the first independent claim played a crucial role in enhancing embedding quality. The table below highlights the strengths of various neural architectures.

Transformer-Based and Patent-Specific Models

General-Purpose vs. Domain-Specific Transformers

Transformer models have reshaped patent classification by introducing context-aware word representations, but not all transformers handle patent text equally well. The difference between general-purpose and domain-specific models is particularly noticeable in this field.

General-purpose transformers like BERT, pre-trained on diverse web and book text, offer solid language understanding. However, patent documents combine legal jargon with highly technical scientific language - something general models rarely encounter. Domain-specific models tackle this challenge by training directly on patent data. PatentBERT, for instance, was specifically trained on patent claims and abstracts, enabling it to align with the unique structure and style of patent documents. Interestingly, researchers discovered that patent claims alone often suffice for classification, challenging the belief that full-document input is always necessary.

PatentSBERTa, a model developed by Hamid Bekamiri and colleagues at Aalborg University in September 2024, set a new benchmark by fine-tuning on 1.5 million patent claims. Validated against 663 CPC subclasses across nearly 1.5 million patents, it achieved an F1 score exceeding 66%, making it a standout in large-scale, text-based multi-label classification.

While encoder-based models like PatentSBERTa excel in classifying frequent CPC subclasses, they fall short in handling rare or emerging technology categories. Here, large language models (LLMs) such as LLaMA-3.1 and Qwen-2.5 demonstrate better recall. A January 2026 study using the USPTO-70k dataset confirmed this complementary relationship. Researchers Lorenzo Emer, Marco Lippi, Andrea Mina, and Andrea Vandin highlighted that encoder models are up to 1,000 times more energy-efficient than LLMs during inference, making them highly practical for processing millions of patent documents.

This natural division of strengths has led to a focus on enhancing classification accuracy through enriched inputs and multimodal strategies.

Multimodal and Metadata-Enriched Models

Recent advancements focus on optimizing input data using metadata and retrieval augmentation to improve transformer performance. Adding structured metadata - such as official CPC subclass definitions - has proven especially helpful for classifying rare categories.

One standout approach is Retrieval-Augmented Generation (RAG), which grounds an LLM's prompts in external knowledge retrieved during inference. In the January 2026 USPTO-70k study, researchers applied RAG and LoRA fine-tuning to Qwen-2.5-7B. This metadata-enriched model significantly improved recall for rare technology categories, outperforming standard encoders despite its higher computational demands.

"LLMs - particularly when augmented with few-shot prompting and lightweight retrieval - provide more balanced coverage across the CPC hierarchy and systematically improve recall on rare and infrequent subclasses." - Lorenzo Emer et al.

Dynamic few-shot prompting is another promising technique. Instead of using fixed examples, the model retrieves semantically similar labeled patents from a training set and includes them as context. This approach grounds the model's reasoning in real-world patent examples rather than relying solely on pre-trained knowledge.

A practical advantage: LLMs in the 7B–8B parameter range, such as LLaMA-3.1 and Qwen-2.5, can run on a single GPU with around 16 GB of VRAM. This makes metadata-enriched classification feasible on standard workstation setups, eliminating the need for large-scale infrastructure. However, the trade-offs remain in terms of latency and energy costs. This is why hybrid strategies - using encoders for high-volume, frequent classes and LLMs for rare or exploratory tasks - are increasingly viewed as a balanced solution.

Fine-Tuning BERT for Text Classification (w/ Example Code)

Practical Guidance and Future Directions for Patent Professionals

Encoder Models vs. LLMs for Patent Classification: Key Differences

Choosing the Right Framework for Your Use Case

Selecting the right framework for patent classification hinges on the size and complexity of your portfolio. For portfolios with well-defined technology categories, encoder-based models like PatentSBERTa shine. They offer incredible energy efficiency - up to 1,000× better - and can run smoothly on standard hardware setups. On the other hand, if you’re tackling emerging or niche technologies with limited labeled data, LLMs such as LLaMA-3.1 or Qwen-2.5 are better suited. These models excel in exploratory scenarios and require only a single GPU with around 16 GB of VRAM.

"Encoder-based models offer unmatched efficiency and scalability, whereas LLMs provide complementary semantic flexibility that is most valuable in long-tail and exploratory classification scenarios." - Lorenzo Emer et al.

Improving input quality is essential for better results. Consolidating data into the TAC format and using noun extraction to reduce token counts (without compromising predictive accuracy) can streamline the process.

These strategies are grounded in earlier discussions about efficiency and scalability. While current methods address many classification challenges, ongoing research is opening doors to even more advanced solutions.

Emerging Research Directions

The horizon for patent classification is expanding, with several exciting areas of research pushing the boundaries of what’s possible.

Multimodal models are a leading area of innovation. These models integrate not just text but also visuals like patent drawings and technical diagrams. This is a game changer because many patents rely heavily on visual elements to convey their core ideas - something text-only models often miss entirely.

Another promising area is continual learning. Most models today are static - they’re trained once and then deployed. But as technology evolves and classification systems like CPC are updated, the need for models that can adapt without full retraining is becoming critical. While still in its early stages, research into this adaptability is gaining momentum. Techniques like LoRA (Low-Rank Adaptation) are also helping teams fine-tune large models for specific patent domains without the hefty costs of full-scale retraining.

In a groundbreaking study published in January 2026, Zheng et al. demonstrated the power of combining contrastive pre-training with chain-of-thought reasoning. Using a dataset of 15,000 drone patents, they achieved a Macro-F1 score of 0.847 - a 16% improvement over standard transformer models for underrepresented categories. This research highlights a growing trend: pairing structured reasoning with semantic retrieval to reduce reliance on large, expert-annotated datasets.

These advancements signal a shift toward more adaptable, efficient, and context-aware patent classification systems, paving the way for future breakthroughs.

FAQs

How do I choose between an encoder model and an LLM for CPC/IPC classification?

Deciding whether to use encoder models or LLMs depends on your specific objectives and available resources. Encoder models like BERT and PatentSBERTa are known for their efficiency and perform exceptionally well when it comes to classifying frequent CPC subclasses. On the other hand, LLMs shine in handling rare cases, early-stage technologies, or tasks that span multiple domains.

Many experts opt for a hybrid approach, leveraging the speed and precision of encoders while using LLMs for their ability to tackle more nuanced or specialized tasks. To support such workflows, Patently provides advanced AI tools designed to streamline classification management and ensure optimal results.

What’s the best way to handle long patent claims without losing accuracy?

To tackle lengthy patent claims effectively, advanced models rely on hierarchical analysis. This approach encodes and summarizes entire documents or paragraphs, ensuring the context remains intact while keeping computational demands manageable. Using transformer-based architectures, these models excel at capturing long-range dependencies. Additionally, large language models generate concise summaries that maintain essential semantic details. Patently leverages these techniques to help professionals streamline the processing and classification of intricate patent data.

Which metrics should I track to avoid overrating performance on common classes?

To get dependable results and avoid inflating performance on frequently occurring classes, it’s important to look beyond just aggregate accuracy. Patent datasets often have imbalanced distributions, so metrics like precision, recall, and F1 scores are more reliable indicators. It’s also helpful to monitor coverage error and Label Ranking Average Precision (LRAP). These metrics are particularly useful for tackling the challenges of multi-label classification and addressing the uneven, long-tail distribution common in patent categories.

Related Blog Posts

• Natural Language in Patents: Common Challenges

• Study: NLP Accuracy in Patent Abstract Summaries

• AI Patent Classification: How It Works

• PatentSBERTa vs. Neural Networks: Key Differences