Capturing the workspace state with snapshots
Snapshots capture the present state of a workspace and publish it to the Helix IPLM server without creating a formal release. This section contains details on how to create and use snapshots.
Note: The snapshot capability currently supports built in data management types, including P4 and SVN. DMHandler based data management IPs (git, p4streams) are not yet supported.
Snapshot overview
Snapshots capture the current state of a workspace and publish it to the Helix IPLM server, without the need to make a release. Snapshots can be used to load new workspaces, which are fully functional replicas of the original workspace. Snapshots are a lightweight mechanism to share changes between users without having to create full hierarchical releases.
Some cases where snapshots are useful :
-
A regression has been run in a workspace, and the regression has failed. With highly dynamic projects, with moving aliases and especially resources used @HEAD, there's a significant chance that by the time the regression has finished, the project has evolved enough (some files have been added, some resources have been updated, …) to prevent users from getting an exact copy of the workspace, as seen by the test platform at the time of the regression.
-
Project owners want to keep exact copies of a workspace during the tape-out process so they know exactly what they have delivered at a specific date
-
A user wants to share one of their workspaces, in its current state, with another user, but without creating a hierarchical release of the entire project just for this copy purpose.
- Bug reports and fixes need to be shared between IP providers and consumers prior to creating a new release.
- Facilitate support requests with an exact copy of the workspace.
Snapshot naming
Snapshots are associated with a particular IPV, and are identified by a sequential numbering scheme referenced to the IPV name.
LIBRARY.IP@VERSION#SNAPSHOT.LINE eg tutorial.padring@3#5.TRUNK
The snapshot is associated to the top level IPV in the workspace at the time the snapshot is created. 'tutorial.padring@3#5.TRUNK' indicates the fifth snapshot of the third release of the padring.
Snapshot Contents
A Snapshot records the contents of the workspace and can be loaded into new workspaces. Snapshot contents differ from an IP release in that
- Snapshots capture the set of differences between the IPV hierarchy and the workspace
- Snapshots capture the entire workspace hierarchy, not just a single IP
Note that the workspace captured by a snapshot can be deleted with no loss of data. The snapshots capture the workspace contents and the workspaces can be recreated by loading the snapshot at any time.
A snapshot captures:
- Workspace resource IPVs whose local version is different from the expected version
- Workspace resource IPV whose local alias is different from expected alias. In this case, it capture both the alias and the underlying static version in the workspace
- IPV local files that at different versions from those expected by that IPV
- The full filelist of IPVs @HEAD
> pi snap list --contents
Snapshot tutorial.padring@1#1.TRUNK:
Description - new snapshot
Top IPV - tutorial.padring@1.TRUNK
Resources - tutorial.MS90G@1.TRUNK
tutorial.io5v@3.TRUNK
tutorial.io_tsmc18@1.TRUNK
Owner - admin
Created on - 2020-05-01 18:58:09 -0700 PDT
Workspace - /tmp/workspaces/ws1
Changed resources in Snapshot:
┌───────────────┬─────────┬───┬──────────┐
│ RESOURCES │ PARENTS │ │ SNAPSHOT │
╞═══════════════╪═════════╪═══╪══════════╡
│ tutorial.io5v │ 1.TRUNK │ → │ 3.TRUNK │
└───────────────┴─────────┴───┴──────────┘
Changed files in Snapshot resources:
┌───────────────────────────────────┬──────────┬───┬──────────┐
│ FILES │ RESOURCE │ │ SNAPSHOT │
╞═══════════════════════════════════╪══════════╪═══╪══════════╡
│ tutorial.padring@1.TRUNK │ │ │ │
│ hw_code/test/test.v │ 1 │ → │ 2 │
│ sw_code/test/blg_driver_test.7z │ 1 │ → │ 2 │
└───────────────────────────────────┴──────────┴───┴──────────┘
Found 1 matching object(s).
Snapshots and resources
The workspace can be in a "modified" resources state when the snapshot is captured, either because resources have been updated, or files in the resource have been updated to other versions. For aliased resources, the snapshot will record the static release loaded into the workspace along with the alias. When the Snapshot is loaded into another workspace, the resource will be restored in the new workspace at the alias, but the alias is resolved to the underlying version of the resource at the time the snapshot was created, rather than the latest release of the resource that has that alias attached.
If the alias is no longer set on that version of the resource anymore when the snapshot its loaded into a workspace, the workspace will be created with the static version of the resource, and the alias will not be used. This is consistent with how Helix IPLM currently works when a user loads a workspace with an aliased resource, then the alias on that version of the resource is removed.
Snapshots and permissions
Snapshots created from a partial workspace due to permissions will capture the contents of the workspace only. This means that a snapshot created by user A, then loaded into a workspace by user B when A and B have different permissions, would result in the intersection of the permissions.
Access to the contents of a snapshot is filtered by the permissions of the user account accessing the snapshot, list, diff or other similar commands will not show information on resources or their files to which the user does not have permission.
When creating a snapshot from a workspace with stale permissions (permissions have changed since the workspace was built), If the user has lost permissions to certain resources that are still in the workspace, these resources will be omitted from the snapshot. If the user has gained permissions to additional workspace resources that are not in the workspace, these too will be omitted from the workspace.
Unmanaged, Open, and Missing Data
Any unmanaged or DM ignored files will be silently skipped during the creation of the snapshot
If a resource or a file is 'missing' in the workspace (removed using a unix command without actually deleting from the DM), then this removal is ignored. The snapshot captures the DM's view of the workspace.
Missing resources will be assumed to be in refer mode for the purposes of the snapshot capture, unless a project property resolves to 'mode=local' for that resource.
Files that are open in the IP's DM system (modified but not checked in) must be resolved prior to creating a snapshot from that workspace.
Command Line
The 'pi snapshot' commands are used to manage Snapshots in Helix IPLM:
> pi snap -h
Usage: pi snapshot [-h] SUBCOMMAND ...
Description: Commands related to Snapshots.
Optional arguments:
-h, --help Show this help message and exit
Available sub-commands:
SUBCOMMAND
add Create a new Snapshot.
delete (del, remove, rm)
Delete a Snapshot.
edit Edit description of a Snapshot.
list (ls) List all matching Snapshots.
load Load a Snapshot into a Workspace.
Adding (Creating) Snapshots
Snapshots are created with the 'pi snap add' command. A description of the snapshot must be provided at the time of creation. In a workspace, if the snapshot is created based on the top level IPV in the workspace, a path can alternatively be specified to the workspace that should be captured as a snapshot.
> pi snap add -h
Usage: pi snapshot add [-h] [--description DESCRIPTION] [path]
Description: Create a new Snapshot.
Positional arguments:
path Workspace directory path. Will use path of current
Workspace by default.
Optional arguments:
--description DESCRIPTION, -m DESCRIPTION, -d DESCRIPTION
Short description of the Snapshot.
-h, --help Show this help message and exit
Deleting Snapshots
Snapshots are deleted with the 'pi snap delete' command. By default Helix IPLM will ask for permission before snapshot deletion, provide the '–yes' argument to skip through the confirmation prompt.
> pi snap delete -h Usage: pi snapshot delete [-h] [--yes] identifier Description: Delete a Snapshot. Positional arguments: identifier Identifier of the Snapshot to be deleted. Optional arguments: --yes, -y Skip confirmation prompt when deleting a Snapshot. -h, --help Show this help message and exit
Editing Snapshots
The 'pi snap edit' command is used to modify the description field of the snapshot, after it has been created. A template can be used to define the description if desired.
> pi snap edit -h
Usage: pi snapshot edit [-h] [--template TEMPLATE] identifier
Description: Edit description of a Snapshot.
Positional arguments:
identifier The name of the Snapshot being edited.
Optional arguments:
--template TEMPLATE, -t TEMPLATE
Use the specified template to edit the Snapshot. The
template is an ASCII file that defines the Snapshot.
This can be used to edit Snapshots with a script and
avoid the interactive editor. To read the template
from STDIN, set TEMPLATE='-'.
-h, --help Show this help message and exit
Listing Snapshots
The 'pi snap list' command is used to list snapshots.
> pi snap list -h
Usage: pi snapshot list [-h] [--format {csv,json,long,table} | --verbose]
[--contents] [--model MODEL] [--all] [--max MAX]
[--query QUERY | --squery SQUERY]
[identifier [identifier ...]]
Description: List all matching Snapshots. A variety of format and filtering
options are supported.
Positional arguments:
identifier Filter Snapshots by name.
Optional arguments:
--all, -a Include all Snapshots without filtering by creator
(shows only Snapshots created by the current user
otherwise).
--contents Display a list of all resources and files, and the
version of each file for the specified Snapshot. This
can generate a LOT of output if a Snapshot contains a
large number of files.
--format {csv,json,long,table}
Return the results using the specified data structure.
Default value is "table".
--max MAX Include at most MAX IP Versions (IPVs).
--model MODEL Use the specified report model (table report only).
--query QUERY, -q QUERY
Filter using the Helix IPLM Query Language.
--squery SQUERY, -sq SQUERY
Filter using a Saved Helix IPLM Query.
--verbose, -v Alias for --format long.
-h, --help Show this help message and exit
Controlling the Number of Snapshots Displayed
By default only one of the snapshots created by the user will be displayed by 'pi snap list' additional snapshots can be listed with the following options.
| Command Option | Description |
|---|---|
| --all, -a | By default only created by the current user will be returned. This option will show snapshots created by all users. |
| --max MAX | Return at most MAX number of Snapshots on an IPV. --max 0 will display all the Snapshots on the IPV |
Configuring the Output Format
Helix IPLM can display the list output in a number of different formats
| Command Option | Format | Description |
|---|---|---|
| --format | table | The default format option, a list of values in table format. |
| long | Displays detailed information about the Snapshot, equivalent to --verbose, v output | |
| json | Display the list of Snapshots in json format. Combine with --contents option to include the file contents of the Snapshot. Note that the Helix IPLM API should be used for scripting purposes. | |
| csv | Display the list of Snapshots in comma separated value format. | |
| --verbose, -v |
|
Displays detailed information about the Snapshot, equivalent to --format long output |
| --contents |
|
Display file contents of the Snapshot. Add to --format json to include the file list in the JSON output. |
| --model MODEL |
|
Display a custom table format MODEL defined in piclient.conf. See the Client Configuration section for details concerning the configuration of display models. |
Running queries
See the Querying Snapshots in the System section for more details about using queries.
| Command Option | Description |
|---|---|
| --query QUERY, -q QUERY | Search using a Helix IPLM query language expression |
| --squery SQUERY, -sq SQUERY | Search using a saved Helix IPLM query language expression |
Loading snapshots
Snapshots are loaded into new workspaces using the 'pi snapshot load' command. The snapshot must be specified, while the workspace directory to use is optional. If not specified the workspace directory is set to the name of the LIBRARY.IP.
> pi snap load -h
Usage: pi snapshot load [-h] [--args ARGS] snapshot [workspace]
Description: Load a Snapshot into a Workspace.
Positional arguments:
snapshot The Snapshot to load into the Workspace.
workspace The Workspace directory. By default the directory is
set to the name of the IP.
Optional arguments:
--args ARGS Arguments passed to the hooks.
-h, --help Show this help message and exit
Snapshots workflow
A user that makes some changes to files or resources in a workspace can run 'pi snap add -d <DESCRIPTION>' and create a snapshot of that workspace. The snapshot is then added to the list of snapshots on the top IPV of the workspace.
[tau deb8:ws1] pi snap add -d new Successfully created Snapshot 'tutorial.tutorial@5#1.TRUNK'. [tau deb8:ws1] pi snap ls ┌─────────────────────────────┬───────────────────────────┬────────────────────────┬───────────────────────────────┐ │ SNAPSHOT │ TOP-IPV │ PARENT │ DATE │ ╞═════════════════════════════╪═══════════════════════════╪════════════════════════╪═══════════════════════════════╡ │ tutorial.tutorial@5#1.TRUNK │ tutorial.tutorial@5.TRUNK │ /home/vishal/pi_ws/ws1 │ 2019-09-30 11:47:01 -0700 PDT │ └─────────────────────────────┴───────────────────────────┴────────────────────────┴───────────────────────────────┘ Found 1 matching object(s).
Snapshots can be listed using 'pi snap ls' and optionally filtered by IP or IPV.
Snapshots can then be loaded into workspaces by running 'pi snap load <SNAP>'
[tau deb8:pi_ws] pi snap load tutorial.tutorial@5#1.TRUNK s1 Loading Snapshot 'tutorial.tutorial@5#1.TRUNK' into Workspace '/home/vishal/pi_ws/s1'. ┌────────────────────────┬─────────────────┬───────┬──────────────────────┐ │ NAME │ VERSION │ MODE │ RELATIVE PATH │ ╞════════════════════════╪═════════════════╪═══════╪══════════════════════╡ │ tutorial.tutorial │ 5.TRUNK │ Refer │ blocks/tutorial │ │ tutorial.CADenv │ GOLD.TRUNK [@1] │ Refer │ blocks/CADenv │ │ tutorial.MS90G │ 1.TRUNK │ Refer │ blocks/MS90G │ │ tutorial.acells_tsmc18 │ 1.TRUNK │ Refer │ blocks/acells_tsmc18 │ │ tutorial.adc │ HEAD.TRUNK [@1] │ Local │ blocks/adc │ │ tutorial.aes512 │ 1.TRUNK │ Refer │ blocks/aes512 │ │ tutorial.analog_top │ HEAD.TRUNK [@1] │ Local │ blocks/analog_top │ │ tutorial.bist_sram │ 1.TRUNK │ Refer │ blocks/bist_sram │ │ tutorial.clk_mux │ 1.TRUNK │ Refer │ blocks/clk_mux │ │ tutorial.clkgen │ HEAD.TRUNK [@1] │ Local │ blocks/clkgen │ │ tutorial.cpu │ 1.TRUNK │ Refer │ blocks/cpu │ │ tutorial.dbuf │ 1.TRUNK │ Refer │ blocks/dbuf │ │ tutorial.digital_top │ 1.TRUNK │ Refer │ blocks/digital_top │ │ tutorial.events_if │ 1.TRUNK │ Refer │ blocks/events_if │ │ tutorial.flash │ 1.TRUNK │ Refer │ blocks/flash │ │ tutorial.flash_if │ 1.TRUNK │ Refer │ blocks/flash_if │ │ tutorial.gen_dig │ 1.TRUNK │ Refer │ blocks/gen_dig │ │ tutorial.interface │ 1.TRUNK │ Refer │ blocks/interface │ │ tutorial.intf_ana │ HEAD.TRUNK [@1] │ Local │ blocks/intf_ana │ │ tutorial.io5v │ 1.TRUNK │ Refer │ blocks/io5v │ │ tutorial.io_tsmc18 │ 1.TRUNK │ Refer │ blocks/io_tsmc18 │ │ tutorial.laysc_tsmc18 │ 1.TRUNK │ Refer │ blocks/laysc_tsmc18 │ │ tutorial.padring │ 1.TRUNK │ Refer │ blocks/padring │ │ tutorial.proj_tech │ 1.TRUNK │ Refer │ blocks/proj_tech │ │ tutorial.pwr_mgmt_ana │ HEAD.TRUNK [@1] │ Local │ blocks/pwr_mgmt_ana │ │ tutorial.rx_channel │ 1.TRUNK │ Refer │ blocks/rx_channel │ │ tutorial.rxtx │ 1.TRUNK │ Refer │ blocks/rxtx │ │ tutorial.stup_ana │ HEAD.TRUNK [@1] │ Local │ blocks/stup_ana │ │ tutorial.sys_bus │ 1.TRUNK │ Refer │ blocks/sys_bus │ │ tutorial.t0 │ 1.TRUNK │ Refer │ blocks/t0 │ │ tutorial.t1 │ 1.TRUNK │ Refer │ blocks/t1 │ │ tutorial.timers │ 1.TRUNK │ Refer │ blocks/timers │ │ tutorial.trc │ HEAD.TRUNK [@1] │ Local │ blocks/trc │ │ tutorial.verif_config │ 1.TRUNK │ Refer │ blocks/verif_config │ └────────────────────────┴─────────────────┴───────┴──────────────────────┘
This string recreates the workspace status as closely as possible to when the snapshot was taken. All the dynamic blocks (@HEAD, @LATEST, @ALIAS etc.) continue to be dynamic, but are loaded as they were when the snapshot was taken.
This workspace is now ready to be used like a normal workspace to run tests or do further development, or run updates and generate releases etc.
