Create a Neo4j graph and explore it

This page explains how to leverage Neo4j to explore your Datashare projects.

Prerequisites

We recommend using a recent release of Datashare (>= 14.0.0) to use this feature. To download a specific version, click on 'All platforms and versions' here.

If you are not familiar with graph and Neo4j, take a look at the following resources:

• Get started with Neo4j

• Find out what is a graph database?

• Learn Neo4j fundamentals

• Check out how to use Neo4j for investigative journalism

The documents and entities graph

Neo4j is a graph database technology which lets you represent your data as a graph.

Inside Datashare, Neo4j lets you connect entities between them through documents in which they appear.

After creating a graph from your Datashare project, you will be able to explore this graph and visualize these kinds of relationships between you project entities:

In the above graph, we can see 3 e-mail document nodes in orange, 3 e-mail address nodes in red, 1 person node in green and 1 location node in yellow. Reading the relationship types on the arrows, we can deduce the following information from the graph:

• [e-mail] emailed [e-mail], the sent email has an ID starting with f4db344...

• One person named vincent is mentioned inside this email, as well as the california location

• Finally, the e-mail also mentions the [e-mail] e-mail address which is also mentioned in 2 other e-mail documents (with ID starting with 11df197... and 033b4a2...)

Graph nodes

The Neo4j graph is composed of :Document nodes representing Datashare documents and :NamedEntity nodes representing entities mentioned in these documents.

The :NamedEntity nodes are additionally annotated with their entity types: :NamedEntity:PERSON, :NamedEntity:ORGANIZATION, :NamedEntity:LOCATION, :NamedEntity:EMAIL...

Graph relationships

In most cases, an entity :APPEARS_IN a document, which means that it was detected in the document content. In the particular case of e-mail documents and EMAIL addresses, it is most of the time possible to identify richer relationships from the e-mail metadata, such as who sent (:SENT relationship) and who received (:RECEIVED relationship) the e-mail.

When an :EMAIL address entity is neither :SENT or :RECEIVED, like it is the case in the above graph for [e-mail], it means that the address was mentioned in the e-mail document body.

When a document is embedded inside another document (as an e-mail attachment for instance), the child document is connected to its parent through the :HAS_PARENT relationship.

Create your Datashare project's graph

The creation of a Neo4j graph inside Datashare is supported through a plugin. To use the plugin to create a graph, follow these instructions:

• When using Datashare on your computer

• When Datashare is running on your server

After the graph is created, open the menu, go to the 'Projects' page, select your project and go to the Graph tab.

You should be able to visualize a new Neo4j widget displaying the number of documents and entities found inside the graph:

Access your project's graph

Depending on your access to the Neo4j database behind Datashare, you might need to export the Neo4j graph and import it locally to access it from visualization tools.

Exporting and importing the graph into your own database is also useful when you want to perform write operations on your graph without any consequences on Datashare.

With read access to Datashare's Neo4j database

If you have read access to the Neo4j database (it should be the case if you are running Datashare on your computer), you will be able to plug visualization tools to it and start exploring.

Without read access to Datashare's Neo4j database

If you can't have read access to the database, you will need to export it and import it into your own Neo4j instance (running on your laptop for instance).

Ask for a DB dump

If it's possible, ask you system administrator for a DB dump obtained using the neo4j-admin database dump command.

Export your graph from Datashare

In case you don't have access to the DB and can't be provided with a dump, you can export the graph from inside. Be aware that limits might be applied on the size of the exported graph.

To export the graph, open the menu, click 'Projects' > 'All projects' > select your project > open the Graph tab. At step 2 called 'Format', select the 'Cypher shell' export format and at the end of the form, click the 'Export' button:

In case you want to restrict the size of the exported graph, you can restrict the export to a subset of documents and their entities using, at step 3, the 'Filters' 'Paths' and 'File types'.

DB import

Depending on how you run Neo4j on your laptop, use one of the following ways to import your graph into your DB:

Docker

• Identify your Neo4j instance container ID:

  docker ps | grep neo4j # Should display your running neo4j container ID

• Copy your the graph dump inside your Neo4j container import directory:

  docker cp \
      <export-path> \
      <neo4j-container-id>:/var/lib/neo4j/imports/datashare-graph.dump

• Import the dumped file using the cypher-shell command:

  docker exec -it <neo4j-container-id> /bin/bash
  ./bin/cypher-shell -f imports/datashare-graph.dump 

Neo4j Desktop import

• Open 'Cypher shell':

desktop-shell

• Copy your the graph dump inside your neo4j instance import directory:

  cp <export-path> imports

• Import the dumped file using the cypher-shell command:

  ./bin/cypher-shell -f imports/datashare-graph.dump 

You will now be able to explore the graph imported in your own Neo4j instance.

Explore and visualize entity links

Once your graph is created and you can access it (see this section if you can't access the Datashare's Neo4j instance), you will be able to use your favorite tool to extract meaningful information from it.

Connect to your database

Once you access your Neo4j database, you can use different tools to visualize and explore it. You can start by connecting the Neo4j Desktop to your DB.

Visualize and explore with Neo4j Bloom

Neo4j Bloom is a simple and powerful tool developed by Neo4j to quickly visualize and query graphs, if you run Neo4j Enterprise Edition. Bloom lets you navigate and explore the graph through a user interface similar to the one below:

bloom-viz

Neo4j Bloom is accessible from inside Neo4j Desktop app.

Find out more information about how to use Neo4j Bloom to explore your graph with:

• Bloom's User Guide

• Bloom's Quick Start

• This series of videos about graph exploration with Bloom

Query the graph with Neo4j Browser

The Neo4j Browser lets you run Cypher queries on your graph to explore it and retrieve information from it. Cypher is like SQL for graphs, running Cypher queries inside the Neo4j browser lets you explore the results as shown below:

browser-viz

The Neo4j Browser is available for both Enterprise and Community distributions. You can access it:

• Inside the Neo4j Desktop app when running Neo4j from the Desktop app

• At http://localhost:7474/browser/ when running Neo4j inside Docker

Visualize and explore with Linkurious Enterprise Explorer

Linkurious is a proprietary software which, similarly to Neo4j Bloom, lets you visualize and query your graph through a powerful UI.

Find out more information about Linkurious:

• Linkurious User Manual

• configure Linkurious with neo4j

• run Linkurious inside Docker

Visualize with Gephi

Gephi is a simple open-source visualization software. It is possible to export graphs from Datashare into the GraphML File Format and import them into Gephi.

Find out more information about:

• How to export your graph in the GraphML format

• Gephi features

• How to get started with Gephi

Export your graph in the GraphML format

To export the graph in the GraphML file format, open the menu, click 'Projects' > 'All projects' > select your project > open the Graph tab. At step 2 called 'Format', select the 'Graph ML' export format and at the end of the form, click the 'Export' button:

In case you want to restrict the size of the exported graph, you can restrict the export to a subset of documents and their entities using, at step 3, the 'Filters' 'Paths' and 'File types'.

You will now be able to visualize the graph using Gephi by opening the exported GraphML file in it.