Command-line interface

FrameTree's command line interface allows you to create stores and frame sets, add source and sink columns, apply pipelines and create derivatives. It can be grouped into three sections, Store management, frame specification, and processing.

Store management

Commands used to access remove data stores and save them for further use

frametree store add

Saves the details for a new data store in the configuration file ('~/.frametree/stores.yaml').

TYPE of the storage class, typically the name of the package the type is defined, e.g. 'xnat' or 'bids'. More specific types can be given by using a colon to separate the package name and type, e.g. frametree.xnat:XnatViaCS

NAME The name the store will be saved as in the configuration file. This is used to refer to the store when using the frametree CLI.

frametree store add [OPTIONS] TYPE NAME

Options

-s, --server <server>

The URI of the server to connect to (if applicable)

-u, --user <user>

The username to use to connect to the store

-p, --password <password>

The password to use to connect to the store

-c, --cache <cache>

The location of a cache dir to download local copies of remote data

-d, --race-condition-delay <race_condition_delay>

How long to wait for changes on a incomplete download before assuming it has been interrupted, clearing it and starting again

-o, --option <name> <value>

Additional key-word arguments that are passed to the store class

Arguments

TYPE

Required argument

NAME

Required argument

frametree store rename

Renames a store in the configuration file

OLD_KNAME The current name of the store.

NEW_NAME The new name for the store.

frametree store rename [OPTIONS] OLD_NAME NEW_NAME

Arguments

OLD_NAME

Required argument

NEW_NAME

Required argument

frametree store remove

Remove a saved data store from the config file

NAME The name the store was given when its details were saved

frametree store remove [OPTIONS] NAME

Arguments

NAME

Required argument

frametree store refresh

Refreshes credentials saved for the given store (typically a token that expires)

NAME The name the store was given when its details were saved

frametree store refresh [OPTIONS] NAME

Options

-u, --user <user>

The username to use to connect to the store

-p, --password <password>

The password to use to connect to the store

Arguments

NAME

Required argument

frametree store ls

List available stores that have been saved

frametree store ls [OPTIONS]

Frame specification

Commands used to define and work with datasets

frametree define

Define the tree structure and IDs to include in a frame set. Where possible, the definition file is saved inside the dataset for use by multiple users, if not possible it is stored in the ~/.frametree directory.

ADDRESS string containing the nick-name of the store, the ID of the dataset (e.g. XNAT project ID or file-system directory) and the frame tree's name in the format <store-nickname>//<dataset-id>[@<frametree-name>]

HIERARCHY the data frequencies that are present in the data tree. For some store types this is fixed (e.g. XNAT-> subject > session) but for more flexible (e.g. MockRemote), which dimension each layer of sub-directories corresponds to can be arbitrarily specified. dimensions

frametree define [OPTIONS] ADDRESS [HIERARCHY]...

Options

--axes <axes>

The enum that specifies the axes of the FrameTree. Defaults to Clinical, which consists of the typical dataset>group>subject>session data tree used in medimage trials/studies

--include <freq-id>

The rows to include in the dataset. First value is the row-frequency of the ID (e.g. 'group', 'subject', 'session') followed by the IDs to be included in the dataset. If the second arg contains '/' then it is interpreted as the path to a text file containing a list of IDs

--exclude <freq-id>

The rows to exclude from the dataset. First value is the row-frequency of the ID (e.g. 'group', 'subject', 'session') followed by the IDs to be included in the dataset. If the second arg contains '/' then it is interpreted as the path to a text file containing a list of IDs

--id-pattern <row-frequency> <pattern>

Specifies how IDs of row frequencies that not explicitly provided are inferred from the IDs that are. For example, given a set of subject IDs that are a combination of the ID of the group that they belong to + their member IDs (i.e. matched test/controls have same member ID), e.g.

CONTROL01, CONTROL02, CONTROL03, ... and TEST01, TEST02, TEST03

the group ID can be extracted by providing the ID to source it from (i.e. subject) and a regular expression (in Python regex syntax: https://docs.python.org/3/library/re.html) with a single group corresponding to the inferred IDs

--id-pattern group 'subject:([A-Z]+)[0-9]+' --id-pattern member 'subject:[A-Z]+([0-9]+)'

Arguments

ADDRESS

Required argument

HIERARCHY

Optional argument(s)

frametree add-source

Adds a source column to a dataset. A source column selects comparable items along a dimension of the dataset to serve as an input to pipelines and analyses.

ADDRESS The path to the dataset including store and dataset name (where applicable), e.g. central-xnat//MYXNATPROJECT:pass_t1w_qc

NAME: The name the source will be referenced by

FORMAT: The data type of the column. Can be a field (int|float|str|bool), field array (ty.List[int|float|str|bool]) or "file-set" (file, file+header/side-cars or directory)

frametree add-source [OPTIONS] ADDRESS NAME DATATYPE

Options

-f, --row-frequency <dimension>

The row-frequency that items appear in the dataset (e.g. per 'session', 'subject', 'visit', 'group', 'dataset' for common:Clinical data dimensions

Default:

'highest'

-p, --path <path>

Path to item in the dataset. If 'regex' option is provided it will be treated as a regular-expression (in Python syntax)

--order <order>

If multiple items match the remaining criteria within a session, select the <order>th of the matching items

-q, --quality <quality>

For data stores that enable flagging of data quality, this option can filter out poor quality scans

--regex, --no-regex

Whether the 'path' option should be treated as a regular expression or not

-h, --header <key-val>

Match on specific header value. This option is only valid for select formats that the implement the 'header_val()' method (e.g. medimage/dicom-series).

Arguments

ADDRESS

Required argument

NAME

Required argument

DATATYPE

Required argument

frametree add-sink

Adds a sink column to a dataset. A sink column specifies how data should be written into the dataset.

Arguments

dataset_path

The path to the dataset including store and dataset name (where applicable), e.g. central-xnat//MYXNATPROJECT:pass_t1w_qc

name

The name the source will be referenced by

datatype

The data type of the column. Can be a field (int|float|str|bool), field array (ty.List[int|float|str|bool]) or "file-set" (file, file+header/side-cars or directory)

frametree add-sink [OPTIONS] ADDRESS NAME DATATYPE

Options

-f, --row-frequency <dimension>

The row-frequency that items appear in the dataset (e.g. per 'session', 'subject', 'visit', 'group', 'dataset' for Clinical data dimensions

Default:

'highest'

-p, --path <path>

Path to item in the dataset. If 'regex' option is provided it will be treated as a regular-expression (in Python syntax)

-s, --salience <salience>

The salience of the column, i.e. whether it will show up on 'frametree derive menu'

Arguments

ADDRESS

Required argument

NAME

Required argument

DATATYPE

Required argument

frametree missing-items

Finds the IDs of rows that are missing a valid entry for an item in the column.

ADDRESS of the dataset including store and dataset name (where

applicable), e.g. central-xnat//MYXNATPROJECT:pass_t1w_qc

COLUMN_NAMES, [COLUMN_NAMES, ...] for the columns to check, defaults to all source columns

frametree missing-items [OPTIONS] ADDRESS [COLUMN_NAMES]...

Arguments

ADDRESS

Required argument

COLUMN_NAMES

Optional argument(s)

Processing

Commands for applying workflows and analyses to framesets and generate derivative data

frametree apply

Apply a Pydra workflow to a dataset as a pipeline between two columns

ADDRESS string containing the nickname of the data store, the ID of the dataset (e.g. XNAT project ID or file-system directory) and the dataset's name in the format <store-nickname>//<dataset-id>[@<dataset-name>]

PIPELINE_NAME is the name of the pipeline

WORKFLOW_LOCATION is the location to a Pydra workflow on the Python system path, <MODULE>:<WORKFLOW>

frametree apply [OPTIONS] ADDRESS PIPELINE_NAME WORKFLOW_LOCATION

Options

-i, --input <col-name> <pydra-field> <required-datatype>

the link between a column and an input of the workflow. The required format is the location (<module-path>:<class>) of the format expected by the workflow

-o, --output <col-name> <pydra-field> <produced-datatype>

the link between an output of the workflow and a sink column. The produced datatype is the location (<module-path>:<class>) of the datatype produced by the workflow

-p, --parameter <name> <value>

a fixed parameter of the workflow to set when applying it

-s, --source <col-name> <pydra-field> <required-datatype>

add a source to the dataset and link it to an input of the workflow in a single step. The source column must be able to be specified by its path alone and be already in the datatype required by the workflow

-k, --sink <col-name> <pydra-field> <produced-datatype>

add a sink to the dataset and link it to an output of the workflow in a single step. The sink column be in the same datatype as produced by the workflow

-f, --row-frequency <row_frequency>

the row-frequency of the rows the pipeline will be executed over, i.e. will it be run once per-session, per-subject or per whole dataset, by default the highest row-frequency rows (e.g. per-session)

--overwrite, --no-overwrite

whether to overwrite previous pipelines

Arguments

ADDRESS

Required argument

PIPELINE_NAME

Required argument

WORKFLOW_LOCATION

Required argument

frametree derive

Derive data for a data sink column and all prerequisite columns.

ADDRESS string containing the nickname of the data store, the ID of the dataset (e.g. XNAT project ID or file-system directory) and the dataset's name in the format <store-nickname>//<dataset-id>[@<dataset-name>]

COLUMNS are the names of the sink columns to derive

frametree derive [OPTIONS] ADDRESS [COLUMNS]...

Options

-w, --work <work>

The location of the directory where the working files created during the pipeline execution will be stored

--plugin <plugin>

The Pydra plugin with which to process the workflow

--loglevel <loglevel>

The level of detail logging information is presented

Arguments

ADDRESS

Required argument

COLUMNS

Optional argument(s)

frametree menu

Display the potential derivatives that can be derived

frametree menu [OPTIONS]

frametree ignore-diff

Ignore difference between provenance of previously generated derivative and new parameterisation

frametree ignore-diff [OPTIONS]