# Datashare

## Datashare

- [About Datashare](https://icij.gitbook.io/datashare/readme.md): Datashare allows you to search in your files, regardless of their format. It is a free open-source software developed by the International Consortium of Investigative Journalists (ICIJ).
- [Ask for help](https://icij.gitbook.io/datashare/ask-for-help.md)
- [Concepts](https://icij.gitbook.io/datashare/concepts.md): This page list all the concepts implemented by Datashare that users might want to understand before starting to search within documents.
- [Running modes](https://icij.gitbook.io/datashare/concepts/running-modes.md): Datashare runs using different modes with their own features.
- [CLI stages](https://icij.gitbook.io/datashare/concepts/cli-stages.md): When running Datashare from the command-line, pick which "stages" to apply to process your documents.
- [SCAN](https://icij.gitbook.io/datashare/concepts/cli-stages/scan.md): The SCAN stage walks a directory and queues every file it finds.
- [SCANIDX](https://icij.gitbook.io/datashare/concepts/cli-stages/scanidx.md): The SCANIDX stage records the paths already present in an index, so a later run can skip them.
- [DEDUPLICATE](https://icij.gitbook.io/datashare/concepts/cli-stages/deduplicate.md): The DEDUPLICATE stage filters paths that appear more than once in the queue during a run.
- [INDEX](https://icij.gitbook.io/datashare/concepts/cli-stages/index.md): The INDEX stage extracts text, metadata and embedded documents, and writes them to Elasticsearch. It is the expensive one.
- [ENQUEUEIDX](https://icij.gitbook.io/datashare/concepts/cli-stages/enqueueidx.md): The ENQUEUEIDX stage pushes documents that are already indexed back onto a queue, so a later stage can process them.
- [CATEGORIZE](https://icij.gitbook.io/datashare/concepts/cli-stages/categorize.md): The CATEGORIZE stage fills the contentTypeCategory field of documents that are already indexed.
- [NLP](https://icij.gitbook.io/datashare/concepts/cli-stages/nlp.md): The NLP stage extracts named entities from documents that are already indexed.
- [CREATENLPBATCHESFROMIDX](https://icij.gitbook.io/datashare/concepts/cli-stages/createnlpbatchesfromidx.md): The CREATENLPBATCHESFROMIDX stage groups indexed documents into batches and submits one named entity recognition task per batch.
- [BATCHNLP](https://icij.gitbook.io/datashare/concepts/cli-stages/batchnlp.md): BATCHNLP exists in the stage list but is not runnable from the command line.
- [ARTIFACT](https://icij.gitbook.io/datashare/concepts/cli-stages/artifact.md): The ARTIFACT stage caches the payload of embedded documents on disk, so previews and downloads do not re-parse their container.
- [CLI reference](https://icij.gitbook.io/datashare/concepts/cli-reference.md): Complete reference of Datashare's command-line interface subcommands and global flags.
- [About the local mode](https://icij.gitbook.io/datashare/local-mode/about-the-local-mode.md)
- [Embedded mode](https://icij.gitbook.io/datashare/local-mode/about-the-local-mode/embedded-mode.md)
- [Install on Mac](https://icij.gitbook.io/datashare/local-mode/install-datashare-on-mac.md): These pages will help you set up and install Datashare on your computer.
- [Install Datashare](https://icij.gitbook.io/datashare/local-mode/install-datashare-on-mac/install-datashare.md)
- [Start Datashare](https://icij.gitbook.io/datashare/local-mode/install-datashare-on-mac/open-datashare-on-mac.md): Find the Datashare application on your computer and run it locally on your browser.
- [Add documents to Datashare](https://icij.gitbook.io/datashare/local-mode/install-datashare-on-mac/add-documents-to-datashare-on-mac.md): Datashare provides a folder on your Mac to collect documents you want to have in Datashare.
- [Install on Windows](https://icij.gitbook.io/datashare/local-mode/install-datashare-on-windows.md): These pages will help you set up and install Datashare on your computer.
- [Install Datashare](https://icij.gitbook.io/datashare/local-mode/install-datashare-on-windows/install-datashare.md): You must have Windows 7 Service Pack 2 or any newer version.
- [Start Datashare](https://icij.gitbook.io/datashare/local-mode/install-datashare-on-windows/open-datashare-on-windows.md): Find the application on your computer and run it locally in your browser.
- [Add documents to Datashare](https://icij.gitbook.io/datashare/local-mode/install-datashare-on-windows/add-documents-to-datashare-on-windows.md): Datashare provides a folder to collect documents on your computer to index in Datashare.
- [Install on Linux](https://icij.gitbook.io/datashare/local-mode/install-datashare-on-linux.md): These pages will help you set up and install Datashare on your computer.
- [Install Datashare](https://icij.gitbook.io/datashare/local-mode/install-datashare-on-linux/install-datashare.md)
- [Start Datashare](https://icij.gitbook.io/datashare/local-mode/install-datashare-on-linux/open-datashare-on-linux.md): Find the application on your computer and run it locally on your browser.
- [Add documents to Datashare](https://icij.gitbook.io/datashare/local-mode/install-datashare-on-linux/add-documents-to-datashare-on-linux.md): Datashare provides a folder to collect documents on your computer to index in Datashare.
- [Install with Docker](https://icij.gitbook.io/datashare/local-mode/install-with-docker.md): This page will help you set up and install Datashare within a Docker.
- [Find entities](https://icij.gitbook.io/datashare/local-mode/find-entities.md): This page helps you find entities (people, organizations, locations, e-mail addresses) in your documents.
- [Add more languages](https://icij.gitbook.io/datashare/local-mode/add-more-languages.md): This page explains how to install language packages to support Optical Character Recognition (OCR) on more languages.
- [Install plugins and extensions](https://icij.gitbook.io/datashare/local-mode/plugins-and-extensions.md): This page explains how to locally add plugins and extensions to Datashare.
- [Neo4j](https://icij.gitbook.io/datashare/local-mode/create-the-neo4j-graph.md): This page explains how to setup Neo4j, install the Neo4j plugin and create a graph on your computer.
- [Install Neo4j plugin](https://icij.gitbook.io/datashare/local-mode/create-the-neo4j-graph/run-datashare-with-the-neo4j-plugin.md)
- [Create and update Neo4j graph](https://icij.gitbook.io/datashare/local-mode/create-the-neo4j-graph/create-and-update-the-graph.md): This page describes how to create and maintain your neo4j graph up to date with your computer's Datashare projects
- [About the server mode](https://icij.gitbook.io/datashare/server-mode/about-the-server-mode.md)
- [Install with Docker](https://icij.gitbook.io/datashare/server-mode/install-with-docker.md): This page explain how to start Datashare within a Docker in server mode.
- [Permission model](https://icij.gitbook.io/datashare/server-mode/permission-model.md): Hierarchical RBAC authorization model based on Casbin (instance/domain/project scopes).
- [Add documents from the CLI](https://icij.gitbook.io/datashare/server-mode/add-documents-from-the-cli.md): The shortest path from an empty project to searchable documents on a server install.
- [Indexing](https://icij.gitbook.io/datashare/server-mode/indexing.md): How Datashare turns a directory of files into a searchable index, and how to drive that process from the command line on your server.
- [Indexing scenarios](https://icij.gitbook.io/datashare/server-mode/indexing/scenarios.md): Copy-paste recipes for the indexing situations you will actually meet on a server: first pass, incremental updates, resume, distribution and backfills.
- [Indexing options](https://icij.gitbook.io/datashare/server-mode/indexing/options.md): Every option that changes how Datashare indexes: command-line flags, environment variables and what cannot be configured from the command line.
- [Tuning and performance](https://icij.gitbook.io/datashare/server-mode/indexing/tuning.md): How to make indexing fast: CPU, OCR threads, memory, Elasticsearch, and the specific case of mail archives.
- [Troubleshooting indexing](https://icij.gitbook.io/datashare/server-mode/indexing/troubleshooting.md): What the errors in your indexing logs mean, which ones are harmless, and what to do about the ones that are not.
- [Add entities from the CLI](https://icij.gitbook.io/datashare/server-mode/add-entities-from-the-cli.md)
- [Manage users from the CLI](https://icij.gitbook.io/datashare/server-mode/manage-users-from-the-cli.md): Create and delete Datashare users with the datashare user subcommands.
- [Manage projects from the CLI](https://icij.gitbook.io/datashare/server-mode/manage-projects-from-the-cli.md): Create and delete projects, and grant or revoke project roles, with the datashare project subcommands.
- [User management](https://icij.gitbook.io/datashare/server-mode/user-management.md)
- [OAuth2](https://icij.gitbook.io/datashare/server-mode/user-management/oauth2.md): OAuth2 authentication with a third-party identity provider
- [HTML form](https://icij.gitbook.io/datashare/server-mode/user-management/form-auth.md): HTML form authentication with credentials stored in a database or Redis.
- [Basic with a database](https://icij.gitbook.io/datashare/server-mode/user-management/basic-with-a-database.md): Basic authentication with a database.
- [Basic with Redis](https://icij.gitbook.io/datashare/server-mode/user-management/basic-with-redis.md): Basic authentication with Redis
- [Dummy](https://icij.gitbook.io/datashare/server-mode/user-management/dummy.md): Dummy authentication provider that disables authentication.
- [Neo4j](https://icij.gitbook.io/datashare/server-mode/create-the-neo4j-graph.md): This page explains how to setup Neo4j, install the Neo4j plugin and create a graph on your server
- [Install Neo4j plugin](https://icij.gitbook.io/datashare/server-mode/create-the-neo4j-graph/run-datashare-with-the-neo4j-plugin.md)
- [Create and update Neo4j graph](https://icij.gitbook.io/datashare/server-mode/create-the-neo4j-graph/create-and-update-the-graph.md): This page describes how to create and maintain your Neo4j graph up to date with your server's Datashare projects
- [Performance considerations](https://icij.gitbook.io/datashare/server-mode/performance-considerations.md): A short summary of what makes Datashare fast or slow, and where the detailed guidance lives.
- [Search projects](https://icij.gitbook.io/datashare/usage/search-projects.md): Projects are collections of documents. Datashare displays statistics about each projects.
- [Explore a project](https://icij.gitbook.io/datashare/usage/explore-a-project.md): A project is a collection of documents. Datashare displays statistics about each projects.
- [Search documents](https://icij.gitbook.io/datashare/usage/search-documents.md): Search with the main search bar and configure settings to display your search's results.
- [Search with operators or Regex](https://icij.gitbook.io/datashare/usage/search-with-operators.md): To make your searches more precise, use operators in the main search bar.
- [Filter documents](https://icij.gitbook.io/datashare/usage/filter-documents.md): Filters are on the left of the search bar. You can contextualize, exclude, lock and reset them. Active filters are displayed in the search breadcrumb.
- [Explore a document](https://icij.gitbook.io/datashare/usage/explore-a-document.md): Explore the document's data through different tabs.
- [Batch search documents](https://icij.gitbook.io/datashare/usage/batch-search-documents.md): Batch searches allow to get the results of each query of a list all at once: instead of searching each query one by one, upload a list, set options/filters and see the matching documents.
- [Star, tag and recommend](https://icij.gitbook.io/datashare/usage/star-documents.md): Turn the documents into starred, tag them or, in server mode, recommend them to project's other members.
- [Keyboard shortcuts](https://icij.gitbook.io/datashare/usage/use-keyboard-shortcuts.md): Shortcuts help do some actions faster.
- [Create a Neo4j graph and explore it](https://icij.gitbook.io/datashare/usage/explore-the-neo4j-graph.md): This page explains how to leverage Neo4j to explore your Datashare projects.
- [FAQ](https://icij.gitbook.io/datashare/usage/faq.md): 👷‍♀️ This page is currently being written by Datashare team.
- [General](https://icij.gitbook.io/datashare/usage/faq/general.md): 👷‍♀️ This page is currently being written by Datashare team.
- [Can I use Datashare with no internet connection?](https://icij.gitbook.io/datashare/usage/faq/general/can-i-use-datashare-with-no-internet-connection.md)
- [Can I download a document from Datashare?](https://icij.gitbook.io/datashare/usage/faq/general/can-i-download-a-document.md)
- [Can I remove document(s) from Datashare?](https://icij.gitbook.io/datashare/usage/faq/general/can-i-remove-a-document-from-datashare.md)
- [Do you recommend OS or machines for large corpuses?](https://icij.gitbook.io/datashare/usage/faq/general/do-you-recommend-os-or-machines-for-large-corpuses.md)
- [Can I use an external drive as data source?](https://icij.gitbook.io/datashare/usage/faq/general/can-i-use-an-external-drive-as-data-source.md)
- [How can we use Datashare on a collaborative mode on a server?](https://icij.gitbook.io/datashare/usage/faq/general/how-can-we-use-datashare-on-a-collaborative-mode-on-a-server.md)
- [What should I do if I get more than 10,000 results?](https://icij.gitbook.io/datashare/usage/faq/general/what-should-i-do-if-i-get-more-than-10-000-results.md)
- [Why results from a simple search and a batch search can be slightly different?](https://icij.gitbook.io/datashare/usage/faq/general/why-results-from-a-simple-search-and-a-batch-search-can-be-sligthly-different.md)
- [How can I uninstall Datashare?](https://icij.gitbook.io/datashare/usage/faq/general/how-can-i-uninstall-datashare.md)
- [Advanced: how can I do bulk actions with Tarentula?](https://icij.gitbook.io/datashare/usage/faq/general/advanced-how-can-i-do-bulk-actions-with-tarentula.md)
- [How to run Neo4j?](https://icij.gitbook.io/datashare/usage/faq/general/how-to-run-neo4j.md): This page explains how to run a Neo4j instance inside docker. For any additional information please refer to the Neo4j documentation: https://neo4j.com/docs/getting-started/
- [How can I contact ICIJ for help, bug reporting or suggestions?](https://icij.gitbook.io/datashare/usage/faq/general/how-can-i-contact-icij-for-help-bug-reporting-or-suggestions.md)
- [Definitions](https://icij.gitbook.io/datashare/usage/faq/definitions.md): 👷‍♀️ This page is currently being written by Datashare team.
- [What is an entity?](https://icij.gitbook.io/datashare/usage/faq/definitions/what-is-a-named-entity.md)
- [What are NLP pipelines?](https://icij.gitbook.io/datashare/usage/faq/definitions/what-are-nlp-pipelines.md)
- [What is fuzziness?](https://icij.gitbook.io/datashare/usage/faq/definitions/what-is-fuzziness.md)
- [What are proximity searches?](https://icij.gitbook.io/datashare/usage/faq/definitions/what-are-proximity-searches.md)
- [Common errors](https://icij.gitbook.io/datashare/usage/faq/common-errors.md): 👷‍♀️ This page is currently being written by Datashare team.
- ['We were unable to perform your search.' What should I do?](https://icij.gitbook.io/datashare/usage/faq/common-errors/your-search-query-is-wrong.md): This can be due to some syntax errors in the way you wrote your query.‌
- [List of common errors leading to "failure" in Batch Searches](https://icij.gitbook.io/datashare/usage/faq/common-errors/list-of-common-errors-leading-to-failure-in-batch-searches.md)
- [What if Datashare says 'No documents found'?](https://icij.gitbook.io/datashare/usage/faq/common-errors/it-says-no-documents-found.md)
- [What if tasks are 'running' but not completing?](https://icij.gitbook.io/datashare/usage/faq/common-errors/tasks-are-running-but-not-completing.md)
- [What if the 'View' of my documents is 'not available'?](https://icij.gitbook.io/datashare/usage/faq/common-errors/i-dont-see-any-preview-of-my-document..md)
- [What do I do if Datashare opens a blank screen in my browser?](https://icij.gitbook.io/datashare/usage/faq/common-errors/datashare-opens-a-blank-screen-in-my-browser..md)
- [I see entities in the filters but not in the documents](https://icij.gitbook.io/datashare/usage/faq/common-errors/i-see-people-organizations-and-locations-in-the-filters-but-not-in-the-documents.md)
- [Datashare doesn't open. What should I do?](https://icij.gitbook.io/datashare/usage/faq/common-errors/datashare-doesnt-open.-what-should-i-do.md)
- [How to contribute](https://icij.gitbook.io/datashare/developers/how-to-contribute.md): 👷‍♀️ This page is currently being written by Datashare team.
- [Backend](https://icij.gitbook.io/datashare/developers/backend.md)
- [API](https://icij.gitbook.io/datashare/developers/backend/api.md): The Datashare API is fully defined using the OpenAPI 3.0 specification and automatically generated after every Datashare release.
- [Database Schema](https://icij.gitbook.io/datashare/developers/backend/database.md)
- [Write extensions](https://icij.gitbook.io/datashare/developers/backend/write-extensions.md)
- [Frontend](https://icij.gitbook.io/datashare/developers/frontend.md)
- [Design System](https://icij.gitbook.io/datashare/developers/frontend/design-system.md)
- [Write plugins](https://icij.gitbook.io/datashare/developers/frontend/write-plugins.md): What if you want to integrate text translations to Datashare’s interface? Or make it display tweets scraped with Twint? Ask no more: there is plugins for that!
- [CLI with Tarentula](https://icij.gitbook.io/datashare/developers/introduction-to-tarentula.md): Datashare Tarentula is a powerful command-line toolbelt designed to streamline bulk operations against any Datashare instance.
- [Script with Playground](https://icij.gitbook.io/datashare/developers/introduction-to-playground.md): Datashare Playground delivers a collection of Bash scripts (free of external dependencies) that streamline interaction with a Datashare instance’s Elasticsearch index and Redis queue.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information, you can query the documentation dynamically by asking a question.
Perform an HTTP GET request on a page URL with the `ask` query parameter:
```
GET https://icij.gitbook.io/datashare/readme.md?ask=<question>
```
The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.
Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.