Capturing the workspace state with snapshots

Snapshots capture the present state of a workspace and publish it to the Helix IPLM server without creating a formal release. This section contains details on how to create and use snapshots.

Note: The snapshot capability currently supports built in data management types, including P4 and SVN. DMHandler based data management IPs (git, p4streams) are not yet supported.

Snapshot overview

Snapshots capture the current state of a workspace and publish it to the Helix IPLM server, without the need to make a release. Snapshots can be used to load new workspaces, which are fully functional replicas of the original workspace. Snapshots are a lightweight mechanism to share changes between users without having to create full hierarchical releases.

Some cases where snapshots are useful :

A regression has been run in a workspace, and the regression has failed. With highly dynamic projects, with moving aliases and especially resources used @HEAD, there's a significant chance that by the time the regression has finished, the project has evolved enough (some files have been added, some resources have been updated, …) to prevent users from getting an exact copy of the workspace, as seen by the test platform at the time of the regression.
Project owners want to keep exact copies of a workspace during the tape-out process so they know exactly what they have delivered at a specific date
A user wants to share one of their workspaces, in its current state, with another user, but without creating a hierarchical release of the entire project just for this copy purpose.
Bug reports and fixes need to be shared between IP providers and consumers prior to creating a new release.
Facilitate support requests with an exact copy of the workspace.

Snapshot naming

Snapshots are associated with a particular IPV, and are identified by a sequential numbering scheme referenced to the IPV name.

Snapshot Example

LIBRARY.IP@VERSION#SNAPSHOT.LINE
eg
tutorial.padring@3#5.TRUNK

The snapshot is associated to the top level IPV in the workspace at the time the snapshot is created. 'tutorial.padring@3#5.TRUNK' indicates the fifth snapshot of the third release of the padring.

Snapshot Contents

A Snapshot records the contents of the workspace and can be loaded into new workspaces. Snapshot contents differ from an IP release in that

Snapshots capture the set of differences between the IPV hierarchy and the workspace
Snapshots capture the entire workspace hierarchy, not just a single IP

Note that the workspace captured by a snapshot can be deleted with no loss of data. The snapshots capture the workspace contents and the workspaces can be recreated by loading the snapshot at any time.

A snapshot captures:

Workspace resource IPVs whose local version is different from the expected version
Workspace resource IPV whose local alias is different from expected alias. In this case, it capture both the alias and the underlying static version in the workspace
IPV local files that at different versions from those expected by that IPV
The full filelist of IPVs @HEAD

Snapshot Example Contents

> pi snap list --contents
Snapshot tutorial.padring@1#1.TRUNK:
    Description               - new snapshot
    Top IPV                   - tutorial.padring@1.TRUNK
    Resources                 - tutorial.MS90G@1.TRUNK
                                tutorial.io5v@3.TRUNK
                                tutorial.io_tsmc18@1.TRUNK
    Owner                     - admin
    Created on                - 2020-05-01 18:58:09 -0700 PDT
    Workspace                 - /tmp/workspaces/ws1

  Changed resources in Snapshot:
    ┌───────────────┬─────────┬───┬──────────┐
    │ RESOURCES     │ PARENTS │   │ SNAPSHOT │
    ╞═══════════════╪═════════╪═══╪══════════╡
    │ tutorial.io5v │ 1.TRUNK │ → │ 3.TRUNK  │
    └───────────────┴─────────┴───┴──────────┘

  Changed files in Snapshot resources:
    ┌───────────────────────────────────┬──────────┬───┬──────────┐
    │ FILES                             │ RESOURCE │   │ SNAPSHOT │
    ╞═══════════════════════════════════╪══════════╪═══╪══════════╡
    │ tutorial.padring@1.TRUNK          │          │   │          │
    │   hw_code/test/test.v             │    1     │ → │    2     │
    │   sw_code/test/blg_driver_test.7z │    1     │ → │    2     │
    └───────────────────────────────────┴──────────┴───┴──────────┘

Found 1 matching object(s).

Snapshots and resources

The workspace can be in a "modified" resources state when the snapshot is captured, either because resources have been updated, or files in the resource have been updated to other versions. For aliased resources, the snapshot will record the static release loaded into the workspace along with the alias. When the Snapshot is loaded into another workspace, the resource will be restored in the new workspace at the alias, but the alias is resolved to the underlying version of the resource at the time the snapshot was created, rather than the latest release of the resource that has that alias attached.

If the alias is no longer set on that version of the resource anymore when the snapshot its loaded into a workspace, the workspace will be created with the static version of the resource, and the alias will not be used. This is consistent with how Helix IPLM currently works when a user loads a workspace with an aliased resource, then the alias on that version of the resource is removed.

Snapshots and workspace views

If a workspace has a workspace view attached when a snapshot is created, the workspace version of files that are included in the workspace (not filtered by the view) is captured in the Snapshot. The files excluded from the workspace by the view iscaptured at the server version. Revision patterns are ignored.

Snapshots and permissions

Snapshots created from a partial workspace due to permissions will capture the contents of the workspace only. This means that a snapshot created by user A, then loaded into a workspace by user B when A and B have different permissions, would result in the intersection of the permissions.

Access to the contents of a snapshot is filtered by the permissions of the user account accessing the snapshot, list, diff or other similar commands will not show information on resources or their files to which the user does not have permission.

When creating a snapshot from a workspace with stale permissions (permissions have changed since the workspace was built), If the user has lost permissions to certain resources that are still in the workspace, these resources will be omitted from the snapshot. If the user has gained permissions to additional workspace resources that are not in the workspace, these too will be omitted from the workspace.

Unmanaged, Open, and Missing Data

Any unmanaged or DM ignored files will be silently skipped during the creation of the snapshot

If a resource or a file is 'missing' in the workspace (removed using a unix command without actually deleting from the DM), then this removal is ignored. The snapshot captures the DM's view of the workspace.

Missing resources will be assumed to be in refer mode for the purposes of the snapshot capture, unless a project property resolves to 'mode=local' for that resource.

Files that are open in the IP's DM system (modified but not checked in) must be resolved prior to creating a snapshot from that workspace.

Command Line

The 'pi snapshot' commands are used to manage Snapshots in Helix IPLM:

pi snapshot Command

> pi snap -h
Usage: pi snapshot [-h] SUBCOMMAND ...

Description: Commands related to Snapshots.

Optional arguments:
  -h, --help            Show this help message and exit

Available sub-commands:
  SUBCOMMAND
    add                 Create a new Snapshot.
    delete (del, remove, rm)
                        Delete a Snapshot.
    edit                Edit description of a Snapshot.
    list (ls)           List all matching Snapshots.
    load                Load a Snapshot into a Workspace.

Adding (Creating) Snapshots

Snapshots are created with the 'pi snap add' command. A description of the snapshot must be provided at the time of creation. In a workspace, if the snapshot is created based on the top level IPV in the workspace, a path can alternatively be specified to the workspace that should be captured as a snapshot.

pi snap add Command

> pi snap add -h
Usage: pi snapshot add [-h] [--description DESCRIPTION] [path]

Description: Create a new Snapshot.

Positional arguments:
  path                  Workspace directory path. Will use path of current
                        Workspace by default.

Optional arguments:
  --description DESCRIPTION, -m DESCRIPTION, -d DESCRIPTION
                        Short description of the Snapshot.
  -h, --help            Show this help message and exit

Deleting Snapshots

Snapshots are deleted with the 'pi snap delete' command. By default Helix IPLM will ask for permission before snapshot deletion, provide the '–yes' argument to skip through the confirmation prompt.

pi snap delete Command

> pi snap delete -h
Usage: pi snapshot delete [-h] [--yes] identifier

Description: Delete a Snapshot.

Positional arguments:
  identifier  Identifier of the Snapshot to be deleted.

Optional arguments:
  --yes, -y   Skip confirmation prompt when deleting a Snapshot.
  -h, --help  Show this help message and exit

Editing Snapshots

The 'pi snap edit' command is used to modify the description field of the snapshot, after it has been created. A template can be used to define the description if desired.

pi snap edit Command

> pi snap edit -h
Usage: pi snapshot edit [-h] [--template TEMPLATE] identifier

Description: Edit description of a Snapshot.

Positional arguments:
  identifier            The name of the Snapshot being edited.

Optional arguments:
  --template TEMPLATE, -t TEMPLATE
                        Use the specified template to edit the Snapshot. The
                        template is an ASCII file that defines the Snapshot.
                        This can be used to edit Snapshots with a script and
                        avoid the interactive editor. To read the template
                        from STDIN, set TEMPLATE='-'.
  -h, --help            Show this help message and exit

Listing Snapshots

The 'pi snap list' command is used to list snapshots.

pi snap list Command

> pi snap list -h
Usage: pi snapshot list [-h] [--format {csv,json,long,table} | --verbose]
                        [--contents] [--model MODEL] [--all] [--max MAX]
                        [--query QUERY | --squery SQUERY]
                        [identifier [identifier ...]]

Description: List all matching Snapshots. A variety of format and filtering
options are supported.

Positional arguments:
  identifier            Filter Snapshots by name.

Optional arguments:
  --all, -a             Include all Snapshots without filtering by creator
                        (shows only Snapshots created by the current user
                        otherwise).
  --contents            Display a list of all resources and files, and the
                        version of each file for the specified Snapshot. This
                        can generate a LOT of output if a Snapshot contains a
                        large number of files.
  --format {csv,json,long,table}
                        Return the results using the specified data structure.
                        Default value is "table".
  --max MAX             Include at most MAX IP Versions (IPVs).
  --model MODEL         Use the specified report model (table report only).
  --query QUERY, -q QUERY
                        Filter using the Helix IPLM Query Language.
  --squery SQUERY, -sq SQUERY
                        Filter using a Saved Helix IPLM Query.
  --verbose, -v         Alias for --format long.
  -h, --help            Show this help message and exit

Controlling the Number of Snapshots Displayed

By default only one of the snapshots created by the user will be displayed by 'pi snap list' additional snapshots can be listed with the following options.

Command Option	Description
--all, -a	By default only created by the current user will be returned. This option will show snapshots created by all users.
--max MAX	Return at most MAX number of Snapshots on an IPV. --max 0 will display all the Snapshots on the IPV

Configuring the Output Format

Helix IPLM can display the list output in a number of different formats

Command Option	Format	Description
--format	table	The default format option, a list of values in table format.
	long	Displays detailed information about the Snapshot, equivalent to --verbose, v output
	json	Display the list of Snapshots in json format. Combine with --contents option to include the file contents of the Snapshot. Note that the Helix IPLM API should be used for scripting purposes.
	csv	Display the list of Snapshots in comma separated value format.
--verbose, -v		Displays detailed information about the Snapshot, equivalent to --format long output
--contents		Display file contents of the Snapshot. Add to --format json to include the file list in the JSON output.
--model MODEL		Display a custom table format MODEL defined in piclient.conf. See the Client Configuration section for details concerning the configuration of display models.

Running queries

See the Querying Snapshots in the System section for more details about using queries.

Command Option	Description
--query QUERY, -q QUERY	Search using a Helix IPLM query language expression
--squery SQUERY, -sq SQUERY	Search using a saved Helix IPLM query language expression

Loading snapshots

Snapshots are loaded into new workspaces using the 'pi snapshot load' command. The snapshot must be specified, while the workspace directory to use is optional. If not specified the workspace directory is set to the name of the LIBRARY.IP. A workspace view can be applied at the time of workspace load.

pi snap load Command

> pi snap load -h
Usage: pi snapshot load [-h] [--args ARGS] [--view VIEW] snapshot [workspace]

Description: Load a Snapshot into a Workspace.

Positional arguments:
  snapshot              The Snapshot to load into the Workspace.
  workspace             The Workspace directory. By default the directory is
                        set to the name of the IP.

Optional arguments:
  --args ARGS           Arguments passed to the hooks.
  --view VIEW, -v VIEW  Load the Workspace using the specified View (PWM
                        only).
  -h, --help            Show this help message and exit

Snapshots workflow

A user that makes some changes to files or resources in a workspace can run 'pi snap add -d <DESCRIPTION>' and create a snapshot of that workspace. The snapshot is then added to the list of snapshots on the top IPV of the workspace.

pi snap add -d Command

[tau deb8:ws1] pi snap add -d new
Successfully created Snapshot 'tutorial.tutorial@5#1.TRUNK'.
[tau deb8:ws1] pi snap ls
┌─────────────────────────────┬───────────────────────────┬────────────────────────┬───────────────────────────────┐
│ SNAPSHOT                    │ TOP-IPV                   │ PARENT                 │ DATE                          │
╞═════════════════════════════╪═══════════════════════════╪════════════════════════╪═══════════════════════════════╡
│ tutorial.tutorial@5#1.TRUNK │ tutorial.tutorial@5.TRUNK │ /home/vishal/pi_ws/ws1 │ 2019-09-30 11:47:01 -0700 PDT │
└─────────────────────────────┴───────────────────────────┴────────────────────────┴───────────────────────────────┘
Found 1 matching object(s).

Snapshots can be listed using 'pi snap ls' and optionally filtered by IP or IPV.

Snapshots can then be loaded into workspaces by running 'pi snap load <SNAP>'

pi snap load Command

[tau deb8:pi_ws] pi snap load tutorial.tutorial@5#1.TRUNK s1
Loading Snapshot 'tutorial.tutorial@5#1.TRUNK' into Workspace '/home/vishal/pi_ws/s1'.
┌────────────────────────┬─────────────────┬───────┬──────────────────────┐
│ NAME                   │     VERSION     │  MODE │ RELATIVE PATH        │
╞════════════════════════╪═════════════════╪═══════╪══════════════════════╡
│ tutorial.tutorial      │     5.TRUNK     │ Refer │ blocks/tutorial      │
│ tutorial.CADenv        │ GOLD.TRUNK [@1] │ Refer │ blocks/CADenv        │
│ tutorial.MS90G         │     1.TRUNK     │ Refer │ blocks/MS90G         │
│ tutorial.acells_tsmc18 │     1.TRUNK     │ Refer │ blocks/acells_tsmc18 │
│ tutorial.adc           │ HEAD.TRUNK [@1] │ Local │ blocks/adc           │
│ tutorial.aes512        │     1.TRUNK     │ Refer │ blocks/aes512        │
│ tutorial.analog_top    │ HEAD.TRUNK [@1] │ Local │ blocks/analog_top    │
│ tutorial.bist_sram     │     1.TRUNK     │ Refer │ blocks/bist_sram     │
│ tutorial.clk_mux       │     1.TRUNK     │ Refer │ blocks/clk_mux       │
│ tutorial.clkgen        │ HEAD.TRUNK [@1] │ Local │ blocks/clkgen        │
│ tutorial.cpu           │     1.TRUNK     │ Refer │ blocks/cpu           │
│ tutorial.dbuf          │     1.TRUNK     │ Refer │ blocks/dbuf          │
│ tutorial.digital_top   │     1.TRUNK     │ Refer │ blocks/digital_top   │
│ tutorial.events_if     │     1.TRUNK     │ Refer │ blocks/events_if     │
│ tutorial.flash         │     1.TRUNK     │ Refer │ blocks/flash         │
│ tutorial.flash_if      │     1.TRUNK     │ Refer │ blocks/flash_if      │
│ tutorial.gen_dig       │     1.TRUNK     │ Refer │ blocks/gen_dig       │
│ tutorial.interface     │     1.TRUNK     │ Refer │ blocks/interface     │
│ tutorial.intf_ana      │ HEAD.TRUNK [@1] │ Local │ blocks/intf_ana      │
│ tutorial.io5v          │     1.TRUNK     │ Refer │ blocks/io5v          │
│ tutorial.io_tsmc18     │     1.TRUNK     │ Refer │ blocks/io_tsmc18     │
│ tutorial.laysc_tsmc18  │     1.TRUNK     │ Refer │ blocks/laysc_tsmc18  │
│ tutorial.padring       │     1.TRUNK     │ Refer │ blocks/padring       │
│ tutorial.proj_tech     │     1.TRUNK     │ Refer │ blocks/proj_tech     │
│ tutorial.pwr_mgmt_ana  │ HEAD.TRUNK [@1] │ Local │ blocks/pwr_mgmt_ana  │
│ tutorial.rx_channel    │     1.TRUNK     │ Refer │ blocks/rx_channel    │
│ tutorial.rxtx          │     1.TRUNK     │ Refer │ blocks/rxtx          │
│ tutorial.stup_ana      │ HEAD.TRUNK [@1] │ Local │ blocks/stup_ana      │
│ tutorial.sys_bus       │     1.TRUNK     │ Refer │ blocks/sys_bus       │
│ tutorial.t0            │     1.TRUNK     │ Refer │ blocks/t0            │
│ tutorial.t1            │     1.TRUNK     │ Refer │ blocks/t1            │
│ tutorial.timers        │     1.TRUNK     │ Refer │ blocks/timers        │
│ tutorial.trc           │ HEAD.TRUNK [@1] │ Local │ blocks/trc           │
│ tutorial.verif_config  │     1.TRUNK     │ Refer │ blocks/verif_config  │
└────────────────────────┴─────────────────┴───────┴──────────────────────┘

This string recreates the workspace status as closely as possible to when the snapshot was taken. All the dynamic blocks (@HEAD, @LATEST, @ALIAS etc.) continue to be dynamic, but are loaded as they were when the snapshot was taken.

This workspace is now ready to be used like a normal workspace to run tests or do further development, or run updates and generate releases etc.