p4 storage

Display, verify, or update physical archive storage.

Syntax

p4 [g-opts] storage [-v [-q]] [-c change] [-T tags -F filters] [-m max] archive ...
p4 [g-opts] storage -u [-c change] [-T tags -F filters] [-m max] archive ...
p4 storage -d [-c change] [-y] [-D secs] [-t target] [-q] archive ...
p4 storage -w
p4 storage -U [-q] [-n threads] [--server-size-rcs] archive...
p4 storage -l start [ --nonlbr [-D secs] [-y] [-t targetDirectory] ] //depotdirectory/...
p4 storage -l  { pause | restart | wait } //depotdirectory/...
p4 storage -l  status [--nonlbr] //depotdirectory/...
p4 storage -l cancel //depotdirectory/...

Syntax conventions

Description

To display information about the server archive files and their usage:

p4 storage [-v [-q]] [-c change] [-T tags -F filters] [-m max] archive...

The -l option provides a group of commands to locate orphaned archive files and mark them for removal by the storage -d command.

86400 seconds (24 hours) is the default delay before file removals. To change this default, see -d under Options.

The p4 storage -l {pause | restart | wait} //depotdirectory/... syntax means pause or restart or wait must be chosen. Omit the curly braces ({}) when entering the command.

Fields

The fields that storage displays are:

lbrFile the path of the archive file
lbrRev the revision of the archive file
lbrType

the file type of the archive file, such as text+C

See File types.

lbrRefCount how many revisions refer to the archive file
headTime timestamp of when the record was written
filesize size of the revision contained in the archive file
digest MD5 digest of the revision contained in the archive file
serverSize Size of the archive file on the server at the time that revision was submitted or shelved
compCksum MD5 of the archive file containing the revision on the server if the file is compressed and submit.storagefields is 1

Additional fields when the -v flag is specified:

actualSize actual size of revision contained in the archive file
actualDigest actual MD5 digest of revision contained in the archive file
status outcome of verify, OK, BAD or MISSING

If upgrading from a version prior to 2019.1

If you are upgrading your P4 Server from a version prior to 2019.1 and your Perforce database has many keyed revisions, you might want to avoid the risk of delay incurred by calculating storage digest updates during the upgrade process. If so, consider using the -U option.

Options

Option Description

-v

Verifies that the specified archive files are intact. Computes and displays the digest of each archive.

  • If an archive cannot be reproduced (for example, if the file is missing from the archive),
    the archive's output line ends with MISSING!

  • If there is a saved digest, p4 storage compares it with the computed one.
    If they differ, the output line ends with BAD!

p4 storage -v is similar to p4 verify except that:

  • for storage -v the archive... argument is matched against the lbrPath of the storage tables

  • for p4 verify the archive... is matched against the depotpath field of the revision tables

-u

Computes and saves the digest only for revisions that have no saved digest.

-q

Minimizes the output of the storage -v command, displaying only errors from mismatched digests or unreproducible revisions.

-d

Deletes unreferenced archives. The database row describing the archive is removed as well as the revision of the underlying archive if both of the following are true:

  • The specified archive has a reference count of 0

  • 86400 seconds (default value for 24 hours) has elapsed
    since the storage record was identified by the scanning task
    p4 storage -l start //depotDirectory/...

This expiry time can be adjusted using the lbr.storage.delay configurable and is intended to protect the storage records and archive files of in-progress submits and in-progress shelves from being considered for deletion. To override this configurable, specify the -D secs parameter on the command line. For example, a one minute delay would be
p4 storage -d -D 60 -y //depotDirectory/...

Without the -y as confirmation, the command merely lists the candidate files and revisions it would remove.
With the -y as confirmation, files and revisions as well as zero referenced storage records are removed.

Archives with a reference count of 0 are rare, but can be created by:

  • p4 storage -l start //depotDirectory/...

The listing of the candidate revisions can be suppressed with the -q option.

For an example, see Reclaim disk space on the server.

-t target

 

Filesystem target directory to which orphaned archive files are copied prior to deletion from the directory indicated by
//depotDirectory/...

-c Applies to the specified changelist.
-T

Applies to the specified tags, which can be specified using a comma- or space-delimited list.

Example: -T "lbrFile, lbrRev"

-F

Applies to the specified filter. Lists only records satisfying the filter expression.
This filter syntax is similar to the one used for jobs -e jobview and is used to evaluate the contents of the fields in the preceding list. Filtering is case-sensitive.

Example: -v -F "lbrRefCount = 1 & status=OK"

-m Limits output to the specified number of records

-w

Provides a message that indicates when the upgrade process is complete by displaying
"The storage upgrade process is complete".
For details, see "From 2019.1 or later" under Upgrade the server in the P4 Server Administration Documentation

-U [-q]
[-n threads]
[--server-size-rcs]

 

To avoid the delay incurred by calculating the digest during the upgrade,
consider using the -U option after a storage upgrade has been performed with the lbr.storage.skipkeyed configurable set to skip keyword revisions.

This option reads all the storage records.
When it detects an unset digest field, it recomputes the digest and size fields, then updates the record. Invoking p4 storage -U -q -n threads suppresses progress messages.

The -n threads option specifies the number of threads to use:
0 for no threads, or
n to use the specified number of threads.
If n is not specified, the command uses the lbr.storage.threads configurable to determine the number of threads.

Calculating the serverSize field for RCS file types can be expensive, and so must be explictly requested by giving the
--server-size-rcs option.

-l

 

Allows the administrator to create, pause, restart, wait, and cancel background processes that scans to
locate and mark orphan files.

This process looks for files in the specified
//depotDirectory
and its subdirectories that match the naming convention
for the server's archive file format.
Each file that is found, along with its revision, is checked for a matching entry in the db.storage table.
If no such entry is found, a new record is created that describes the file and revision combination with a reference count of 0.

A later call to p4 storage -d will delete both those records with zero references and
the associated the archive revision.

p4 storage -l has subcommands:

  • start
    Sart a new background depot scanning task starting in the specified //depotdirectory and scan it and all its subdirectories.

    The special //depotdirectory argument of //... is handled by searching for all the local depots listed in the depot table and scanning their top and subdirectories.

    There is no limit on the number of background processes that can be started, subject to the constraints of the hardware and operating system limits.

    Each directory in the archive can only be the subject of a single scan.

    The start command can take an optional --nonlbr argument that will request the scan to remove any files that do not match the file naming convention used by the librarian A subsystem of the server that stores, manages, and provides the archive files to other subsystems of the server. See also 'archive files'. for archive files. Any files that match any of the patterns inthe lbr.scan.ignore_pattern configurable will not be removed. Additionally a -t targetDirectory argument can be given that will request a non-librarian file be moved to the specified targetDirectory rather than be removed. However, without the -y to confirm, no files will be moved or removed, just previewed. For an example, see Locating and removing non-librarian files.

    All files must have aged by lbr.storage.delay seconds to be eligible for removal. The lbr.storage.delay setting can be overriden on the command line by using the -D secs argument.

    If the server shuts down while a scan is unfinished, the scan is marked paused and the administrator can manually restart the scan again when the server is started.

  • pause
    Suspend the named background scan process.

  • restart
    Resume the named paused background scan process.

    If the server shuts down while a scan is unfinished,
    the scan is marked paused so that when the server
    is running again, the administrator can manually restart the scan.

  • wait
    Wait until the named background scan process completes or fails due to an error.

The wait subcommand returns no output.
It only returns when the scanner is no longer active, which means its state is done, error, or paused.
Also, this subcommand waits on a single scanner task.
For example, with //... there is a single scanner task that searches all the local depots.

  • status
    Report on all the background scan processes if the special argument of
    //... is specified, or report on the named background scan process.

    The optional --nonlbr argument displays the deletion status of detected non-librarian files for a particular scan.

    For each scan, the following fields are displayed:

    • scanName
      Name of the scan, the argument passed to the start subcommand, which is the path to scan.
      See the Example of a possible workflow.

    • state
      done, paused, run, busy, or error.

    • dirsProc
      The number of directories that have been processed.

    • filesProc
      The number of files that have been processed.

    • zeroRecs
      The number of zero referenced storage records created. This is the number of orphan revisions that have no matching storage record.

    • dirsErr
      The number of directories skipped due to errors.

    • filesNonLbr
      The number of non librarian files that have been deleted or removed.

    • filesAge
      The number of non librarian files not deleted due to age.

  • cancel
    Remove the scan from the system. A scan remains in the system until it is canceled.
    To re-run a scan, first pause the scan (if it is active), then cancel the scan, even if it has completed or resulted in an error.

Symlinks: Scanning requires that each file only be addressed by a single path under the scanning directory. By default the scanner returns an error if it detects a symlink.
To allow symlinks, set the lbr.storage.allowsymlink configurable to 1.
This should only be done if all the symlinks only reference files and directories
that are not under any scanned directory.

The scanner does not run if the depot is shared between servers.

g-opts

See Global options.

Example of a possible workflow

Locating the orphaned files

To start the scan:

p4 storage -l start //depotDirectory/...

To get the status of the scan:

p4 storage -l status //depotDirectory/...

To pause the scan:

p4 storage -l pause //depotDirectory/...

To end the pause and resume the scan:

p4 storage -l restart //depotDirectory/...

To cancel the scan:

p4 storage -l cancel //depotDirectory/...

A scan remains in the system until it is canceled. To re-run a scan, first pause the scan (if it is active), then cancel the scan, even if it has completed or resulted in an error.

Reclaim disk space on the server

To save the orphaned files to your target directory and delete the zero references along with associated archive files from the depot directory specified:

p4 storage -d -y -t /myTargetDirectory //depotdirectory/...

where /myTargetDirectory is any operating system target directory, and, for Windows, might be something like D:\myCopyOfOrphanedFiles

Locating and removing non-librarian files

To get a preview of non-librarian files:
p4 storage -l start --nonlbr //depot/...

To remove non-librarian files, include -y:
p4 storage -l start --nonlbr -y //depot/...

Usage notes

Can File Arguments Use Revision Specifier? Can File Arguments Use Revision Range? Minimal Access Level Required

No

No

admin, except for the -U option, which requires super.