How Do Generative Image AIs Work? Searching Query Using Scene Graph Part 1.
With the proliferation of image-generative Artificial Intelligence, an increasing number of individuals are expressing a keen interest in understanding the capabilities of graph databases. In particular, they are intrigued by these databases' ability to decode the intricate behaviors of images. Among the various graph analytical functions, Scene graphs play a pivotal role in searching for images and identifying those that bear remarkable similarities. This involves the application of sophisticated algorithms and techniques that empower graph databases not only to grasp the content of images but also to establish meaningful connections between them. In this article, we will delve deeply into the innovative methods that revolutionize image analysis and the retrieval of visual data within the digital landscape,
What is a Scene Graph?
Scene graph is a graph-based analysis method with a new perspective added in the field of computer vision. It represents the attribute information inherent in images or image data as a graph not only of objects, but also relationships between objects. Objects from the scene and their relationships are explicitly expressed using a connection graph. Figure 1. Represents the SPO relationship, that is Subject-Predicate-Object between the scenes.
Searching for Subject-Object-Predicate relationships
To efficiently search for images and objects through Scene Graph, the graph modeling consists of two nodes and edges. The nouns of the image and the object are represented in nodes, the edge that connects the image to the noun is represented as 'eg_connect' edge and an 'eg_predicate' edge represents the noun to the object as a predicate. These connections are shown as a graph in Figure 2.
The 'eg-predicate' edge can be loaded with a list of images as “img_id_arr” to retrieve images from the predicate without unnecessary navigation. When searching for a model using the property information of the graph, the scene graph allows to only search for relationships related to the images and predicate relationships between objects, skipping all unnecessary instances.
Logics behind the Search Queries
Below are the logics of retrieving the SPO corresponding to an image or searching for images that contain SPOs.
Search Query: "Show me the SPO results for image No.1"
-
Search for the noun connected to the 'Image 1' node
-
Find all the 'Noun nodes' connected to the 'Image 1' node
-
Filter the SPOs that contain 'Image 1' in the property of the 'Predicate edge' connected between the 'Noun nodes'.
-
The SPOs corresponding to the 'image 1 node' can be extracted.
In practice, this method can scan for unique predicate edges and thus, yields faster computation compared to a relational database that has to search for all instances.
Search Query: "Show me photos that contain the information 'man (S) driving (P) Car (O)"
-
Perform the SPO search for each noun node and predicate edge
-
List the images stored in the predicate edge property
This method also scans for unique predicate edges and thus, yields faster computation compared to a relational database that has to search for all instances.
Performance of Complex Search Query
A graph database demonstrates its greatest advantages when tasked with searching for intricate SPO relationships that involve multiple hops. The following example illustrates the outcomes and presents the SPO graph corresponding to a query seeking “pictures of a man wearing Nike shoes while playing tennis."
A query to find the path;
(tennis) <- [playing] - (man) - [wearing] -> (shoe) - [has] -> (nike logo)
in the SPO graph will significantly degrade query performance in the relational database because it needs to explore many join relationships.
In this article, we saw the search query logic to retrieve and extract 'Subject - Predicate - Object' between images or objects in an image using Scene Graph. Part.2 of this article will explain how to extract similar images with a graph database's Scene Graph, and how efficient it is to query or extract similar images in contrast to storing data in a traditional relational database.
Stay tuned for the next blog to learn more about the usage of Scene Graphs! If you have any questions or would like to query your data in graph databases, contact us today to get started!