Set up Elasticsearch

You must integrate P4 Search with Elasticsearch. Elasticsearch is the engine used for indexing, searching, and analyzing data.

Requirements

The Elasticsearch requirements for P4 Search are:

Text and metadata are stored in Elasticsearch indices. You must provision sufficient disk space based on the size of your Perforce depot.
The hardware requirements for indexing your data will depend on both the quantity and the type of data you have. Text, 2D images, and 3D models are all handled differently in the back end.
P4 Search requires Elasticsearch 8.12.1 to 8.19.0.

P4 DAM may support different versions of Elasticsearch. For the Elasticsearch version that P4 DAM supports, see the P4 DAM Administrator Guide .

Install Elasticsearch

The following installation steps are applicable to single-node or small site deployments only. To install multiple nodes, see Install Elasticsearch with Debian Package.
Elasticsearch must be installed on a clean computer running the Linux operating system. Ensure that any previous versions of Elasticsearch are removed.

To install Elasticsearch with a Debian package on a single node or in a small site deployment:

Ensure that Elasticsearch is pinned and not upgraded. To pin an Elasticsearch version, follow the instructions for your operating system (OS) distribution:
Ubuntu:
As root, run the following commands:
```
vi /etc/apt/preferences.d/elasticsearch
Package: elasticsearch
Pin: version 8.19.0
Pin-Priority: 999
```
RHEL

Go to the Red Hat Customer Portal and follow the installation process in Restricting a Package to a Fixed Version Number with yum.
Import the Elasticsearch Pretty Good Privacy (PGP) key. Follow the instructions in Import the Elasticsearch PGP Key.
From the Advanced package tool (APT) repository, take the following actions:
1. Install the apt-transport-https package on Debian:
```
sudo apt-get install apt-transport-https
```
2. Save the repository definition to the /etc/apt/sources.list.d/elastic.list file:
```
echo "deb [signed-by=/usr/share/keyrings/elasticsearch-keyring.gpg] https://artifacts.elastic.co/packages/8.x/apt stable main" | sudo tee /etc/apt/sources.list.d/elastic.list
```
3. Install the Elasticsearch Debian package:
```
sudo apt-get update && sudo apt-get install elasticsearch
```
The Elasticsearch password is displayed in your terminal.
Add the Elasticsearch password to the P4 Search configuration file in etc/config.properties file. The user name is elastic.

P4 Search automatically creates the required indexes.
To configure Elasticsearch to start automatically when the system starts, run the following commands:
```
sudo /bin/systemctl daemon-reload 
sudo /bin/systemctl enable elasticsearch.service
```
To start or stop Elasticsearch manually, see Start or stop Elasticsearch.

By default, the Elasticsearch service doesn’t log information in the systemd journal.
(Optional) To enable journalctl logging, remove the --quiet option from the ExecStart command line in the /usr/lib/systemd/system/elasticsearch.service file.

To view logging information in the systemd journal, see View logging information in the systemd journal.

Verify that the Elasticsearch node is running by taking one of the following actions:

Send an HTTPS request to port 9200 on localhost:

curl --cacert /etc/elasticsearch/certs/http_ca.crt -u elastic:$<ELASTIC_PASSWORD> https://localhost:9200

where <ELASTIC_PASSWORD> is the password obtained in the Step 3 of these installation instructions.

If Elasticsearch is running, it sends a response similar to the following example:

{
  "name" : "Cp8oag6",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "AT69_T_DTp-1qgIJlatQqA",
  "version" : {
    "number" : "8.13.1",
    "build_type" : "tar",
    "build_hash" : "f27399d",
    "build_flavor" : "default",
    "build_date" : "2016-03-30T09:51:41.449Z",
    "build_snapshot" : false,
    "lucene_version" : "9.8.0",
    "minimum_wire_compatibility_version" : "1.2.3",
    "minimum_index_compatibility_version" : "1.2.3"
  },
  "tagline" : "You Know, for Search"
}

In a web browser, navigate to the following URL:

https://localhost:9200

Elasticsearch loads its configuration from the /etc/elasticsearch/elasticsearch.yml file by default.

If you are configuring Elasticsearch for large sites, increase the Java Virtual Machine (JVM) memory. Use a custom options file to increase the JVM memory as described in Set JVM options.
To allow P4 Search to work with a self-signed Elasticsearch, do not change the default setting in etc/config.properties file:

com.perforce.p4search.elastic.insecure=true
To add your Elasticsearch certificate to the JRE keystore, see Configure self-signed certificate.

You have successfully installed Elasticsearch. By default, the installation directory is /usr/share/elasticsearch.

The next step is to install the P4 Plugin Filter.

Start or stop Elasticsearch

You can start or stop Elasticsearch as follows:

sudo systemctl start elasticsearch.service
sudo systemctl stop elasticsearch.service

View logging information in the systemd journal

When systemd logging is enabled, you can access logging information by using the journalctl commands:

Tail the journal:
```
sudo journalctl -f
```
Journal entries for the Elasticsearch service:
```
sudo journalctl --unit elasticsearch
```
Journal entries for the Elasticsearch service from a specific start time:
```
sudo journalctl --unit elasticsearch --since  "2016-10-30 18:17:16"
```

For more command line options, run the man journalctl or see journalctl.

Enable logging

Elasticsearch recommends using the Log4j 2 configuration that is shipped by default.

To enable logging, see Elasticsearch application logging.

Install the P4 Plugin Filter

The P4 Plugin Filter limits search results based on the protections table in P4 Server. For more information, see Access Authorization in the P4 Server Administration Documentation.

To install the P4 Plugin Filter:

Import the Perforce package signing key on the server hosting P4 Plugin Filter or P4 Search.

Follow the instructions for your OS distribution:
Ubuntu:
```
wget -qO - https://package.perforce.com/perforce.pubkey | sudo apt-key add -
sudo apt-get update
```
RHEL (run this command as root):

rpm --import https://package.perforce.com/perforce.pubkey

For information about how to verify the authenticity of the signing key, see Perforce packages on the Perforce website.
Configure the Perforce package repository on the server hosting P4 Plugin Filter or P4 Search.

As root, follow the instructions for your OS distribution:

Ubuntu 20.04 LTS:

Create the file /etc/apt/sources.list.d/perforce.list with the following content:

deb https://package.perforce.com/apt/ubuntu/ focal release

Ubuntu 22.04 LTS:

Create the file /etc/apt/sources.list.d/perforce.list with the following content:

deb https://package.perforce.com/apt/ubuntu/ jammy release

Ubuntu 24.04 LTS:

Create the file /etc/apt/sources.list.d/perforce.list with the following content:

deb https://package.perforce.com/apt/ubuntu/ noble release
RHEL 8:
Create the file /etc/yum.repos.d/perforce.repo with the following content:
```
[Perforce]
name=Perforce
baseurl=https://package.perforce.com/yum/rhel/8/x86_64/
enabled=1
gpgcheck=1
```
RHEL 9:
Create the file /etc/yum.repos.d/perforce.repo with the following content:
```
[Perforce]
name=Perforce
baseurl=https://package.perforce.com/yum/rhel/9/x86_64/
enabled=1
gpgcheck=1
```
There are two ways to install the P4 Plugin Filter:
- For Linux, use Install the P4 Plugin Filter from a Linux package.
- For Windows, use Manually install the P4 Plugin Filter.
For information on Elasticsearch plugin installation and testing, see the Elasticsearch plugin documentation.

Install the P4 Plugin Filter from a Linux package

To install the P4 Plugin Filter from a package:
1. Import the Perforce package signing key and configure the Perforce package repository on the server hosting P4 Plugin Filter. To do this, see Install the P4 Plugin Filter.
2. Install the P4 Plugin Filter package on the server hosting Elasticsearch.
  
  Follow the instructions for your OS distribution:
  
  Ubuntu
  
  sudo apt-get install helix-plugin-filter
  
  RHEL (run these commands as root):
  
  yum install helix-plugin-filter
Manually install the P4 Plugin Filter

To manually install the P4 Plugin Filter, also known as the Elasticsearch plugin p4search-filter:
1. Remotely log in to the Elasticsearch host.
2. Change directory so you are in the Elasticsearch installation directory, typically %ES_HOME%\bin.
3. To install the plugin, look in the <p4search-installation>/lib directory for the exact filename. For example, p4search-filter-8.16.1-2024.4.2222222.zip,
4. Install the plugin, either:
  - from a local file, for example:
  - or using the P4 Search endpoint, for example:
5. Restart Elasticsearch after installing the plugin.

Upgrade the P4 Plugin Filter

Follow the instructions for your OS distribution:

Upgrade Elasticsearch

If you upgrade Elasticsearch, you must also upgrade the P4 Plugin Filter to the same version and then restart Elasticsearch. See Upgrade the P4 Plugin Filter.

To upgrade Elasticsearch to a newer release on Ubuntu, do the following:

Unpin your Elasticsearch:

sudo vi /etc/apt/preferences.d/elasticsearch

Verify that the source list for Elasticsearch version 8.x is up to date:
```
sudo vi /etc/apt/sources.list.d/elastic-8.x.list
```
Verify that the Elasticsearch version is listed:
```
apt list -a elasticsearch
```

Upgrade the Elasticsearch Debian package:

sudo apt-get update && sudo apt-get upgrade elasticsearch

Restart the Elasticsearch service:

sudo systemctl start elasticsearch.service

Elasticsearch is now upgraded.

Optional Elasticsearch settings

The following are optional when setting up an Elasticsearch host for P4 Search:

By default, Elasticsearch is accessible only on localhost.

To expose a specific node on the network, set network.host to a different address in the elasticsearch.yml file, as shown in the following example:

network.host: 10.1.2.3

where network.host specifies the IP address of the Elasticsearch server.
By default, Elasticsearch listens for HTTP traffic on the first free port it finds, starting at 9200.

Set http.port to a specific HTTP port, as shown in the following example:

http.port: 9200

where http.port specifies the port number of the Elasticsearch host.

For more information, see the Network module documentation.
To perform discovery when a node is started, pass an initial list of one or more hosts.

For example:

discovery.seed_hosts: ["host1"]

The default list of hosts is ["127.0.0.1", "[::1]"]
You can bootstrap the cluster using an initial set of master-eligible nodes.

For example:

cluster.initial_master_nodes: ["node-1"]

For more information, see the discovery and cluster formation module documentation on Discovery and cluster formation.
In your jvmoptions file, remove the xms and xmx settings. On Ubuntu, this file can be found in:

/usr/share/elasticsearch/config/

For more information about configuring Elasticsearch and the location of the config directory, see Configuring Elasticsearch.