================ DMRIFiberProfile ================ dmrifiberprofile is a tool that analyzes fiber tracts based on DTI volumes or scalar images. .. image:: _static/fiberprofile_ui.png GUI Mode ==================== DTIPlaygroundLab (Web UI) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Run:: $ dmriplaygroundlab Then go to the DMRIFiberProfile menu. In the INPUT tab, enter the file path to the input datasheet containing the paths to the images to be analyzed. You can click the paper clip icon to browse for the file in a GUI. Do the same for selecting the output directory. Next, click on the PROTOCOL tab to define the pipeline. For tract profile extraction, double-click the EXTRACT_Profile module to add it to the pipeline. .. image:: _static/extract_profile_ui.png Fill in the fields in the EXTRACT_Profile module protocol. You **must** provide a comma-separated list of VTK files to profile. If an atlas directory is provided, these file paths may be relative (e.g., `tract1.vtk`). If no atlas directory is provided, the file paths must be absolute (e.g., `/proj/NIRAL/users/alecj/fibers/tract1.vtk`). Check the box for "Input is DTI" if the input images are DTI volumes. If the input images are scalar images, uncheck this box. The program will search for scalar images corresponding to the scalar properties you want to profile. The "Properties to Column Header Map" allows you to map parameters to the column headers in your input datasheet. Default column headers are provided, but they may not match your CSV. If they don't, you can edit the "Column Header" field. For example, if your case IDs are in a column called "Subject ID," you should enter this in the "Column Header" field for the "Case ID" parameter. Once you've entered the requisite fields, you can click the GENERATE PROTOCOLS button to generate the protocol file. This file will be saved in the output directory you specified in the INPUT tab. Finally, click the EXECUTE button to run the pipeline. Alternatively, you can copy the command and run it directly in the terminal.:: $ dmrifiberprofile run -i INPUT_DATASHEET -p OUTPUT_DIR/protocols.yml CLI Mode (Linux/Windows-WSL) ================================ For Windows users, install WSL2 and linux packages with python 3.9 - 3.12 (3.11 recommended). 1. init - Initialize configuration (Default: `$HOME/.niral-dti/dmrifiberprofile-`) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ init command generates the configuration directory and files with following command. One just needs to execute this command only once unless a different configuration is needed. If you want to reset the initial configuration directory, you can run init again.:: $ dmrifiberprofile init If you want to set different config directory other than default one:: $ dmrifiberprofile --config-dir my/config/dir init Once run, `config.yml` and `environment.yml` will be in the directory. You can manually specify the tool directory (which is generated by `install-tools` command) by `--tools-dir` option.:: $ dmrifiberprofile init --tools-dir 2. update - Update if `config.yml` has been changed (e.g. in case of adding user module directory). ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Changing `config.yml` file should be followed by updating `environment.yml` with running update command :: $ dmrifiberprofile [--config-dir my/config/dir] update This will update module-specific informations such as binary locations or package location used by the corresponding module. It simply updates `environment.yml` 3. make-protocols - Generating a default protocol file ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The first thing to do is generate the default protocol file that has pipeline information:: $ dmrifiberprofile [base options] make-protocols -i INPUT_DATASHEET [-o OUTPUT_DIRECTORY] [-d MODULE1 MODULE2 ... ] if `-o` option is omitted, the output protocol will be printed on terminal.`-d` option specifies the list of modules for the analysis, with which command will generate the default pipeline and protocols of the sequence. Same module can be used redundantly. If `-d` option is not specified, the default pipeline will be generated from the file `protocol_template.yml` . You can change the default pipeline in `protocol_template.yml` file. By default, the only module that will run is `EXTRACT_Profile` which extracts the profile of the fiber tracts provided. Modifying the default protocol -------------------------------- If using the `EXTRACT_Profile` module, the default protocol file has two fields that define the tracts, null by default. 1. `atlas` - The directory containing the tracts to be profiled. Must be provided as an absolute path (or with `run --atlas`). 2. `tracts` - The list of tracts to be profiled. Must be provided as a comma-separated list of file names (spaces don't matter), including the .vtk extension. These file names will be concatenated with the path specified by `atlas` to form the full path to the tracts. If empty, all the tracts (`.vtk`) of the `atlas` directory are profiled. Optionally, you can also leave the `atlas` field blank and provide a list of absolute file paths in the `tracts` field. If you try to use this protocol without providing an atlas or tracts, you will receive an error message. Anatomy of EXTRACT_Profile protocol ----------------------------------- Below are the options contained in the `EXTRACT_Profile` protocol. .. code-block:: yaml protocol: atlas: type: directory default_value: null description: Directory containing a set of fiber tracts in the atlas to be analyzed. tracts: type: string default_value: null description: Selected set of tracts to use. Comma-delimited list of file names with .vtk extension included. Each entry must uniquely map to a VTK fiber file in the atlas location. inputIsDTI: type: boolean default_value: true description: Specifies whether the input image is a DTI, and properties are derived from it. propertiesToProfile: type: list default_value: FA, MD, RD, AD description: List of selected properties to profile along tracts. useDisplacementField: type: boolean default_value: true description: Determines whether to convert the image to atlas space using a displacement field. If set to false, the image will not be transformed to atlas space. parameterToColumnHeaderMap: type: dictionary default_value: null description: Optional map of parameters (e.g., scalar names, case id) to column headers to use for each property to profile. resultCaseColumnwise: type: boolean default_value: true description: Specifies whether to store cases as columns instead of rows in the output CSV. planeOfOrigin: type: string candidates: - value: Median description: Origin of profile will be median of tract. - value: CoG description: Origin of profile will be center of gravity. default_value: Median description: Determines the plane that sets the origin of the profile arc length. stepSize: type: integer default_value: 1 description: Specifies how far along the tract to step for each new fiber profile location. supportBandwidth: type: float default_value: 3 description: Standard deviation (mm) of the Gaussian kernel; points within this distance of a sample are averaged. noNaN: type: boolean default_value: false description: Specifies whether to remove fibers with NaN values, used both for FiberPostProcess and DTITractStat. mask: type: file default_value: null description: Optional mask file to use during profile extraction. The mask has to be defined in atlas space. Here's an example of what the `EXTRACT_Profile` protocol might look like with the `atlas` and `tracts` fields filled in: .. code-block:: yaml protocol: atlas: /proj/NIRAL/users/alecjn/test_scripts/DTIPlayground-Tests/tests/input/fiberprofile/atlas inputIsDTI: true mask: null noNaN: false parameterToColumnHeaderMap: FA: FA for original Original DTI Image: Original DTI planeOfOrigin: Median propertiesToProfile: FA, MD resultCaseColumnwise: true stepSize: 1 supportBandwidth: 3 tracts: Arc_L_FrontoParietal-2_extracted_done.vtk, Corpus_Callosum-2_extracted_done.vtk useDisplacementField: true 4. run - Run pipeline ~~~~~~~~~~~~~~~~~~~~~~~~~ To run with existing protocol file:: $ dmrifiberprofile run -i INPUT_DATASHEET -p PROTOCOL_FILE -o OUTPUT_DIR PROTOCOL_FILE is the file generated by `make-protocols` command and appropriately populated with the necessary information. Instead of a datasheet, `-i` can be a folder: the files of every scan below it are detected (grouped by case id, the part of the file names before `_dwi`) and written to `OUTPUT_DIR/datasheet_detected.csv`, and the column map of the protocol is set to match:: $ dmrifiberprofile run -i DATA_FOLDER -p PROTOCOL_FILE -o OUTPUT_DIR With `useDisplacementField: true` (native space) each scan needs its tensor (`_dwi[_QCed]_tensor.nrrd` or `_dwi[_QCed]_DTI.nrrd`) and displacement field (`..._GlobalDisplacementField.nrrd` or `..._DTI_DisplacementField.nrrd`); with `false` (atlas space) its registered tensor (`..._DeformedDTI.nrrd` or `..._DTI_Registered.nrrd`). The maps of the other properties (e.g. `..._NODDI_NDI.nii.gz`, `..._Registered_NDI.nii.gz`) are optional. The run stops with an error listing the known file names if no scan matches. To check the detected datasheet first:: $ dmrifiberprofile make-datasheet DATA_FOLDER -p PROTOCOL_FILE -o datasheet.csv Without a protocol file, the defaults of EXTRACT_Profile are used; `--atlas` sets the atlas folder, and without `tracts` all the tracts (`.vtk`) of the atlas folder are profiled:: $ dmrifiberprofile run -i DATA_FOLDER --atlas ATLAS_FIBERS_FOLDER -o OUTPUT_DIR With `cleanup: noCleanup` (the default), the profile of each scan, tract and property (`.fvp`) is kept with the settings and input files it was computed from; a later run into the same output folder reuses the unchanged ones and only computes the others (e.g. added scans). With `duringProcessing` or `endOfProcessing` everything is recomputed. Analysis and QC of the profiles =============================== Besides running the EXTRACT_Profile pipeline (``dmrifiberprofile run``), ``dmrifiberprofile`` provides the tools that analyse and QC the profiles, ported from the FiberProfileAnalysis scripts. ``dmrifiberprofile --help`` lists all the options of each. ``flip-tensor`` Reflect the tensor frame of a DTI NRRD along axes, for tensors whose components don't match the frame of their header; ``--voxel-frame`` rotates components stored in the frame of the voxel axes. ``detect-tensor-flip`` Find the correction of the tensor frame that orients a DTI correctly, by the coherence of the principal directions along the tracts and, with ``--reference``, their agreement with an atlas tensor. ``parametrize-fibers`` Resample fiber tracts on the arc length grid and store the arc lengths in the fibers, so that the atlas defines them once; replaces ``dtitractstat -f``. ``compute-axis`` 1D axis of the tracts (the average curve per arc length bin), used by ``impute``. ``gather`` Collect the profiles of several runs into one table per tract and metric, ``/_.csv`` (rows: arc length, columns: the scans). ``impute`` Fill the missing profile values with a per-dataset SIREN on the (x, y, z, arc length) of the tract axes; uses a GPU when there is one. ``qc-registration`` QC of the registration of the subjects to the atlas: similarity of the deformed metric maps, angular error, CSF check and an age conditional normative model, with a combined outlier flag. ``--build-normative`` builds that model from a reference cohort, including the mean tensor of each age bin that DTI_Register can register to. ``qc-profiles`` Age binned statistics of the profiles (``/__agebinstats.csv``) and their plots, and the QC of the profiles against prior (normative) statistics, which flags the profiles whose values leave the normative envelope or whose shape doesn't follow it. A typical sequence, from the parametrized atlas fibers to the cleaned profiles:: $ dmrifiberprofile parametrize-fibers Atlas/FibersRaw -o Atlas/FibersParam $ dmrifiberprofile gather --profiles-dir Output_Profiles --fibers-dir Atlas/FibersParam --out-dir Profiles $ dmrifiberprofile compute-axis Atlas/FibersParam -o FiberAxis $ dmrifiberprofile impute --profiles-dir Profiles --axis-dir FiberAxis --out-dir Profiles_Imputed $ dmrifiberprofile qc-registration --data-dir Data --atlas-dir Atlas --normative-dir Atlas/normativeModel --out-dir RegistrationQC $ dmrifiberprofile qc-profiles --profiles-dir Profiles_Imputed --prior-stats-dir Atlas/normProfiles --registration-qc RegistrationQC --clean-dir Profiles_Clean What qc-profiles reads ~~~~~~~~~~~~~~~~~~~~~~ ``--profiles-dir`` takes the tables of ``gather`` (``/_.csv``) or, without gathering them first, the profiles of a single run as EXTRACT_Profile writes them (``//00_EXTRACT_Profile//_.csv``, in either orientation), so one run can be QCed on its own. ``--prior-stats-dir`` takes either layout as well, since the age bin statistics are written next to the tables they were computed from. The age of a profile ~~~~~~~~~~~~~~~~~~~~ The age decides the age bin of a profile. It is the row of the scan in ``--age-csv``, a participants / sessions table (``--age-column`` and ``--age-units`` describe its age column), else ``--age-regex`` on the column name of the profile, ``ses-(\d+)m`` by default. The same table is used by ``qc-registration --age-csv`` and by the ``ageCSV`` option of the DTI_Register module of dmriprep. Profiles with neither are named at the end of the run, as they take part in no age bin. Locations sampled outside the brain ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ A metric that cannot be 0 in tissue (FA, MD, RD, AD, NDI, ODI, ...) is 0 at a position where the fibers of that scan left the brain mask. That is a property of the location, so it is read as missing on every metric of that tract and case, including those where 0 is a valid measurement (``--zero-valid-metrics``, the free water fraction by default); ``--keep-outside-brain`` keeps them, and with ``--clean-dir`` those cells are written empty in the cleaned tables. A profile with less than ``--min-valid-frac`` (0.75) of its positions left is flagged as an outlier, since too little of it is there to judge. See the `README `_ for the remaining options. Development of a new module =========================== Adding a module ~~~~~~~~~~~~~~~ Once initialized, users can add their custom module from scratch or existing system/user modules by following command:: $ dmrifiberprofile add-module [--base-module ] [--edit] Following command will generate initial skeletal files of module:: $ dmrifiberprofile add-module HELLO_World Then you can test if the module can be loaded properly with:: $ dmrifiberprofile update You can use your module right in protocol file. if `-b` , `--base-module` is specified, new model will copy existing code and data from the base module. e.g.:: $ dmrifiberprofile add-module MYFIRST_Module -b EXTRACT_Profile MYFIRST_Module will have same codes and data (module definition yaml file) from EXTRACT_Profile module with new classname and filenames. Developer ~~~~~~~~~ Once module is developed and tested in the user module directory, one can just move that directory in `dtiplayground/dmri/fiberprofile/modules` and commit. Make sure the custom module does not exist in both the user and system module directories. Removing user module ~~~~~~~~~~~~~~~~~~~~ User module can be removed by:: $ dmrifiberprofile remove-module e.g.:: $ dmrifiberprofile remove-module MYFIRST_Module [NOTE] System module cannot be removed by this command. Only user module can be removed. Modules in other directory ~~~~~~~~~~~~~~~~~~~~~~~~~~ You can just copy module directory to `$HOME/.niral-dti/modules/dmrifiberprofile` and check with `$ dmrifiberprofile update` command. Same applies for removal of user modules.