Announcing Figure Extraction: Unlocking generative AI insights from data visualizations in PDF documents
Kensho’s new Figure Extraction feature uses patent-pending AI to pull hundreds of precise data points from charts and visualizations embedded in PDF documents.
Authors: Rob Marvin, Brandon Smock
We’re proud to announce the launch of Figure Extraction, a new Kensho Extract feature that marks a leap forward in our ability to extract data from PDF visualizations at scale with industry-leading accuracy.
Figure Extraction can extract hundreds of data points from data visualizations in PDF documents with far higher numerical accuracy than leading multimodal large language models (LLMs) today. Customers can then easily pull this machine-readable data into additional processing and analysis workflows, or make the textual data available to LLMs and Retrieval-Augmented Generation (RAG) systems for natural language interactions.
What’s new?
Most generative AI applications and document extraction solutions available in the market today struggle to accurately extract numerical data from visualizations like charts, graphs, and plots. We developed this feature to help our customers and teams across our parent company S&P Global extract accurate and meaningful data from figures, unlocking key insights from data they’ve never been able to use before.
Kensho Extract quickly and reliably structures documents, extracts tables and text, and ensures unstructured PDFs are machine-readable for generative AI applications and workflows. Building on Kensho’s core text and table extraction capabilities, Figure Extraction uses a patent-pending approach to help customers uncover insights from charts within their documents. Simply upload your PDF documents, and you’ll receive back highly accurate numerical data in a tabular format for additional processing and analysis.
Kensho Extract supports bar charts at launch. In the example below, we show a scientific bar chart containing six data points along with the table produced by Extract. Other solutions struggle to estimate these numbers accurately, which can disrupt data trends and make it difficult to rely on the result. Kensho Extract, on the other hand, preserves the trends in the data with precise numerical accuracy for each value based on the chart’s resolution.
Kensho’s Figure Extraction capability infers the data values whether they are written in the text of the chart or indicated by the visual elements alone. In fact, it can do both simultaneously. In the next example, the chart below contains a text label of “$66” for Capital Expenditure for the year 2009. However, the bar height indicates the value is actually closer to $64. Kensho Extract returns both — each of these in their own separate column. This versatility automates tedious data extraction while empowering a user to decide what to do with it.
Solving for accurate figure extraction at scale
Much of today’s enterprise data is not machine-readable, particularly when it exists in forms such as data visualizations within PDF documents. However, building data extraction models for numerical figures is a complex task. While multimodal LLMs such as OpenAI’s GPT-4o have attempted to solve for figure extraction, the prevailing model approach is not optimized for the use case. As a result, overall accuracy has been a persistent challenge for current figure extraction solutions in the market.
Our team’s testing found that when applied to figure extraction, current multimodal LLMs commonly produce numerical errors of anywhere from 5 to 25 percent, hallucinate data, and fail to recognize complex charts with more than 20–30 data points. This makes them difficult to rely on for many use cases.
Kensho’s models are specialized for highly accurate figure extraction in complex PDF documents. Our models excel at extracting quantitative data without it being explicitly written in the figure, with the ability to scale to hundreds of data points.
We can’t wait to see what enterprise data our customers unearth with Figure Extraction, and the new use cases and generative AI workflows these insights enable for your organizations. This launch is only the beginning; over the next few months we plan to expand support for additional visualization types including line plots, scatter plots, and pie charts.
Get started
S&P Global customers can head to S&P Global Marketplace to start uncovering AI insights from bar charts in your PDF documents. Figure Extraction is also available directly through the Kensho Extract API and Extract UI.
Want to learn more? Sign up for a free trial or talk to us today!
