Metadata-Version: 2.1
Name: bbrc-validator
Version: 0.5.9
Summary: Systematic sanity checks on imaging datasets within an XNAT environment
Home-page: https://gitlab.com/bbrc/xnat/bbrc-validator
Author: Greg Operto, Jordi Huguet
Author-email: goperto@barcelonabeta.org
License: UNKNOWN
Download-URL: https://gitlab.com/bbrc/xnat/bbrc-validator/-/archive/v0.5.9/bbrc-validator-v0.5.9.tar.gz
Description: bbrc-validator 
        ==============
        
        [![pipeline status](https://gitlab.com/bbrc/xnat/bbrc-validator/badges/master/pipeline.svg)
        ](https://gitlab.com/bbrc/xnat/bbrc-validator/commits/master)
        [![coverage report](https://gitlab.com/bbrc/xnat/bbrc-validator/badges/master/coverage.svg)
        ](https://gitlab.com/bbrc/xnat/bbrc-validator/commits/master)
        [![python](https://img.shields.io/pypi/pyversions/bbrc-validator.svg)
        ](https://pypi.org/project/bbrc-validator)
        [![pypi](https://img.shields.io/pypi/v/bbrc-validator.svg)
        ](https://pypi.org/project/bbrc-validator)
        
        <p align="center">
          <a href="#main-concepts">Main Concepts</a> •
          <a href="#commands">Commands</a> •
          <a href="#example">Examples</a> •
          <a href="#install">Install</a> •
          <a href="#contributing">Contributing</a>
        </p>
        
        **bbrc-validator** is a Python-based software package that performs automatic quality 
        assessment of neuroimaging datasets and their processing derivatives, through 
        collections of "checkpoints". 
        **bbrc-validator** is built on two core concepts: _Tests_ and _Validators_.
        
        - A **Test** checks a specific trait from a given resource (either an imaging 
          session or a single scan). It asks a specific question whose answer can be 
          either `True` or `False` (eg. _"Does this MRI scan have a conversion to NIfTI 
          available?"_). As such, _Tests_ may be seen as [unit tests
          ](https://en.wikipedia.org/wiki/Unit_testing).  A _Test_ class is defined by two 
          attributes (`passing` and `failing`) that refer to two "real-life" cases (one 
          expected to pass the _Test_ and another expected to fail it). In addition, these 
          attributes are systematically used by the [CI
          ](https://en.wikipedia.org/wiki/Continuous_integration) testing.
          
        - A **Validator** is a collection of **Test** objects that may be executed against 
          any [XNAT](https://www.xnat.org/) imaging resource (by referring to their experiment 
          identifiers). Running a _Validator_ on a given experiment takes its associated 
          set of tests, runs them sequentially and collects their results in a JSON object. 
          A human-readable report can be  generated (as a PDF document) with the results
          of the whole procedure.
        
        Main Concepts 
        -------------
        
        - __Test__:
                                        
           ```python
            class MyTest():
                """ Test functionality description """
                passing = 'PASSING_CASE_ID'
                failing = 'FAILING_CASE_ID'
                
                def run():     # executes the Test logic and returns some Results
                    return Results(has_passed=test_outcome, data=some_data)  
                def report():  # provides a human-readable version of Results data
           ```
                         
        - __Validator__:                 
          
           ```python
            class MyValidator():
                def __init__():
                    self.tests = [MyTest, ...]        
                
                def run():     # runs all Tests sequentially             
                def dump():    # compiles all Test results in a single JSON object    
                def report():  # generates a human-readable PDF report based on the results
           ```
                       
        - __Result__: Represents the outcome from the execution of a _Test_. It includes
                      a boolean attribute `has_passed` (representing the outcome of _Test_
                      execution) and some additional `data` object (optionally used for 
                      storing contextual information from the execution).
          
        Commands
        --------
        
        ### `run_validator.py`
        
        Executes the specified _Validator_ against a given image resource (a.k.a XNAT 
        _experiment_) and generates (a) a JSON object with the results of all the Tests 
        and (b) a human-readable PDF report.
        ```
        usage: run_validator.py [-h] --config CONFIG --experiment EXPERIMENT
                                [--validator VALIDATOR] --output OUTPUT [--verbose]
        
        Run a validator against an experiment
        
        optional arguments:
          -h, --help                             show this help message and exit
          --config CONFIG, -c CONFIG             XNAT configuration file
          --experiment EXPERIMENT, -e EXPERIMENT XNAT experiment unique identifier
          --validator VALIDATOR, -v VALIDATOR    Validator name (default:ArchivingValidator)
          --output OUTPUT, -o OUTPUT             PDF file to store the report
          --verbose, -V                          Display verbosal information (optional)
        ```
        
        ### `validation_scores.py`
         
        Given a specific type of _Validator_, collects all results available in an XNAT 
        instance and compiles them in a CSV file.  
        ```
        usage: validation_scores.py [-h] --config CONFIG --version VERSION
                                    [--validator VALIDATOR] --output OUTPUT 
                                    [--project PROJECT] [--verbose]
        
        Compile validation scores
        
        optional arguments:
          -h, --help                    show this help message and exit
          --config CONFIG               XNAT configuration file
          --version VERSION, -v VERSION Filter specific version
          --validator VALIDATOR         Validator name (default:ArchivingValidator)                      
          --output OUTPUT, -o OUTPUT    CSV output file
          --verbose, -V                 Display verbosal information (optional)
        ```
        
        Enables the creation of tables such as the following example obtained from
        `ArchivingValidator` (table trimmed to fit the dimensions of the page).
        
        _Tests_ included:
        
        1. HasUncompressedPixelData
        2. HasCorrectSequences
        3. HasBvecBval
        4. IsClassicDICOM
        5. HasDuplicatedSequences
        6. HasNifti
        7. HasPhilipsPrivateTags
        8. IsStudyDescriptionCorrect
        
        <table border="1" cellpadding="0" cellspacing="0" dir="ltr">
        	<tbody>
        		<tr>
        			<td>Tests</td>
        			<td><pre>#1</pre></td>
        			<td><pre>#2</pre></td>
        			<td><pre>#3</pre></td>
        			<td><pre>#4</pre></td>
        			<td><pre>#5</pre></td>
        			<td><pre>#6</pre></td>
        			<td><pre>#7</pre></td>
        			<td><pre>#8</pre></td>
        		</tr>
        		<tr>
        			<td>Sums</td>
        			<td>11</td>
        			<td>11</td>
        			<td>0</td>
        			<td>11</td>
        			<td>11</td>
        			<td>6</td>
        			<td>0</td>
        			<td>11</td>
        		</tr>
        		<tr>
        			<td>BBRCDEV_E00211</td>
        			<td> ![#c5f015](https://via.placeholder.com/15/c5f015/000000?text=+)</td>
        			<td> ![#c5f015](https://via.placeholder.com/15/c5f015/000000?text=+)</td>
        			<td> ![#f03c15](https://via.placeholder.com/15/f03c15/000000?text=+)</td>
        			<td>![#c5f015](https://via.placeholder.com/15/c5f015/000000?text=+)</td>
        			<td>![#c5f015](https://via.placeholder.com/15/c5f015/000000?text=+)</td>
        			<td>![#f03c15](https://via.placeholder.com/15/f03c15/000000?text=+)</td>
        			<td>![#f03c15](https://via.placeholder.com/15/f03c15/000000?text=+)</td>
        			<td>![#c5f015](https://via.placeholder.com/15/c5f015/000000?text=+)</td>
        		</tr>
        		<tr>
        			<td>BBRCDEV_E00210</td>
        			<td>![#c5f015](https://via.placeholder.com/15/c5f015/000000?text=+)</td>
        			<td>![#c5f015](https://via.placeholder.com/15/c5f015/000000?text=+)</td>
        			<td>![#f03c15](https://via.placeholder.com/15/f03c15/000000?text=+)</td>
        			<td>![#c5f015](https://via.placeholder.com/15/c5f015/000000?text=+)</td>
        			<td>![#c5f015](https://via.placeholder.com/15/c5f015/000000?text=+)</td>
        			<td>![#c5f015](https://via.placeholder.com/15/c5f015/000000?text=+)</td>
        			<td>![#f03c15](https://via.placeholder.com/15/f03c15/000000?text=+)</td>
        			<td>![#c5f015](https://via.placeholder.com/15/c5f015/000000?text=+)</td>
        		</tr>
        		<tr>
        			<td>BBRCDEV_E00213</td>
        			<td>![#c5f015](https://via.placeholder.com/15/c5f015/000000?text=+)</td>
        			<td>![#c5f015](https://via.placeholder.com/15/c5f015/000000?text=+)</td>
        			<td>![#f03c15](https://via.placeholder.com/15/f03c15/000000?text=+)</td>
        			<td>![#c5f015](https://via.placeholder.com/15/c5f015/000000?text=+)</td>
        			<td>![#c5f015](https://via.placeholder.com/15/c5f015/000000?text=+)</td>
        			<td>![#c5f015](https://via.placeholder.com/15/c5f015/000000?text=+)</td>
        			<td>![#f03c15](https://via.placeholder.com/15/f03c15/000000?text=+)</td>
        			<td>![#c5f015](https://via.placeholder.com/15/c5f015/000000?text=+)</td>
        		</tr>
        		<tr>
        			<td>BBRCDEV_E00212</td>
        			<td>![#c5f015](https://via.placeholder.com/15/c5f015/000000?text=+)</td>
        			<td>![#c5f015](https://via.placeholder.com/15/c5f015/000000?text=+)</td>
        			<td>![#f03c15](https://via.placeholder.com/15/f03c15/000000?text=+)</td>
        			<td>![#c5f015](https://via.placeholder.com/15/c5f015/000000?text=+)</td>
        			<td>![#c5f015](https://via.placeholder.com/15/c5f015/000000?text=+)</td>
        			<td>![#c5f015](https://via.placeholder.com/15/c5f015/000000?text=+)</td>
        			<td>![#f03c15](https://via.placeholder.com/15/f03c15/000000?text=+)</td>
        			<td>![#c5f015](https://via.placeholder.com/15/c5f015/000000?text=+)</td>
        		</tr>
        		<tr>
        			<td>BBRCDEV_E00196</td>
        			<td>![#c5f015](https://via.placeholder.com/15/c5f015/000000?text=+)</td>
        			<td>![#c5f015](https://via.placeholder.com/15/c5f015/000000?text=+)</td>
        			<td>![#f03c15](https://via.placeholder.com/15/f03c15/000000?text=+)</td>
        			<td>![#c5f015](https://via.placeholder.com/15/c5f015/000000?text=+)</td>
        			<td>![#c5f015](https://via.placeholder.com/15/c5f015/000000?text=+)</td>
        			<td>![#c5f015](https://via.placeholder.com/15/c5f015/000000?text=+)</td>
        			<td>![#f03c15](https://via.placeholder.com/15/f03c15/000000?text=+)</td>
        			<td>![#c5f015](https://via.placeholder.com/15/c5f015/000000?text=+)</td>
        		</tr>
        		<tr>
        			<td>BBRCDEV_E00214</td>
        			<td>![#c5f015](https://via.placeholder.com/15/c5f015/000000?text=+)</td>
        			<td>![#c5f015](https://via.placeholder.com/15/c5f015/000000?text=+)</td>
        			<td>![#f03c15](https://via.placeholder.com/15/f03c15/000000?text=+)</td>
        			<td>![#c5f015](https://via.placeholder.com/15/c5f015/000000?text=+)</td>
        			<td>![#c5f015](https://via.placeholder.com/15/c5f015/000000?text=+)</td>
        			<td>![#f03c15](https://via.placeholder.com/15/f03c15/000000?text=+)</td>
        			<td>![#f03c15](https://via.placeholder.com/15/f03c15/000000?text=+)</td>
        			<td>![#c5f015](https://via.placeholder.com/15/c5f015/000000?text=+)</td>
        		</tr>
        		<tr>
        			<td>BBRCDEV_E00217</td>
        			<td>![#c5f015](https://via.placeholder.com/15/c5f015/000000?text=+)</td>
        			<td>![#c5f015](https://via.placeholder.com/15/c5f015/000000?text=+)</td>
        			<td>![#f03c15](https://via.placeholder.com/15/f03c15/000000?text=+)</td>
        			<td>![#c5f015](https://via.placeholder.com/15/c5f015/000000?text=+)</td>
        			<td>![#c5f015](https://via.placeholder.com/15/c5f015/000000?text=+)</td>
        			<td>![#f03c15](https://via.placeholder.com/15/f03c15/000000?text=+)</td>
        			<td>![#f03c15](https://via.placeholder.com/15/f03c15/000000?text=+)</td>
        			<td>![#c5f015](https://via.placeholder.com/15/c5f015/000000?text=+)</td>
        		</tr>
        		<tr>
        			<td>BBRCDEV_E00216</td>
        			<td>![#c5f015](https://via.placeholder.com/15/c5f015/000000?text=+)</td>
        			<td>![#c5f015](https://via.placeholder.com/15/c5f015/000000?text=+)</td>
        			<td>![#f03c15](https://via.placeholder.com/15/f03c15/000000?text=+)</td>
        			<td>![#c5f015](https://via.placeholder.com/15/c5f015/000000?text=+)</td>
        			<td>![#c5f015](https://via.placeholder.com/15/c5f015/000000?text=+)</td>
        			<td>![#c5f015](https://via.placeholder.com/15/c5f015/000000?text=+)</td>
        			<td>![#f03c15](https://via.placeholder.com/15/f03c15/000000?text=+)</td>
        			<td>![#c5f015](https://via.placeholder.com/15/c5f015/000000?text=+)</td>
        		</tr>
        		<tr>
        			<td>BBRCDEV_E00219</td>
        			<td>![#c5f015](https://via.placeholder.com/15/c5f015/000000?text=+)</td>
        			<td>![#c5f015](https://via.placeholder.com/15/c5f015/000000?text=+)</td>
        			<td>![#f03c15](https://via.placeholder.com/15/f03c15/000000?text=+)</td>
        			<td>![#c5f015](https://via.placeholder.com/15/c5f015/000000?text=+)</td>
        			<td>![#c5f015](https://via.placeholder.com/15/c5f015/000000?text=+)</td>
        			<td>![#f03c15](https://via.placeholder.com/15/f03c15/000000?text=+)</td>
        			<td>![#f03c15](https://via.placeholder.com/15/f03c15/000000?text=+)</td>
        			<td>![#c5f015](https://via.placeholder.com/15/c5f015/000000?text=+)</td>
        		</tr>
        		<tr>
        			<td>BBRCDEV_E00218</td>
        			<td>![#c5f015](https://via.placeholder.com/15/c5f015/000000?text=+)</td>
        			<td>![#c5f015](https://via.placeholder.com/15/c5f015/000000?text=+)</td>
        			<td>![#f03c15](https://via.placeholder.com/15/f03c15/000000?text=+)</td>
        			<td>![#c5f015](https://via.placeholder.com/15/c5f015/000000?text=+)</td>
        			<td>![#c5f015](https://via.placeholder.com/15/c5f015/000000?text=+)</td>
        			<td>![#f03c15](https://via.placeholder.com/15/f03c15/000000?text=+)</td>
        			<td>![#f03c15](https://via.placeholder.com/15/f03c15/000000?text=+)</td>
        			<td>![#c5f015](https://via.placeholder.com/15/c5f015/000000?text=+)</td>
        		</tr>
        		<tr>
        			<td>BBRCDEV_E00198</td>
        			<td>![#c5f015](https://via.placeholder.com/15/c5f015/000000?text=+)</td>
        			<td>![#c5f015](https://via.placeholder.com/15/c5f015/000000?text=+)</td>
        			<td>![#f03c15](https://via.placeholder.com/15/f03c15/000000?text=+)</td>
        			<td>![#c5f015](https://via.placeholder.com/15/c5f015/000000?text=+)</td>
        			<td>![#c5f015](https://via.placeholder.com/15/c5f015/000000?text=+)</td>
        			<td>![#c5f015](https://via.placeholder.com/15/c5f015/000000?text=+)</td>
        			<td>![#f03c15](https://via.placeholder.com/15/f03c15/000000?text=+)</td>
        			<td>![#c5f015](https://via.placeholder.com/15/c5f015/000000?text=+)</td>
        		</tr>
        	</tbody>
        </table>
        
        Examples
        --------
        
        #### Create a Validator and review the list of its Tests
        1. Set a [pyxnat connection
           ](https://pyxnat.github.io/pyxnat/tutorial.html#setting-up-a-connection)
           to the XNAT instance hosting the data requiring validation.     
        2. Create an instance of `SPM12SegmentValidator`, a _Validator_ for segmentations
           produced using [SPM12 Segment](https://www.fil.ion.ucl.ac.uk/spm/doc/manual.pdf#page=45).
        3. Print out a list of included tests.     
        ```python
        import pyxnat
        intf = pyxnat.Interface(config='.xnat.cfg')
        
        from bbrc.validation import SPM12SegmentValidator
        spmv = SPM12SegmentValidator(lut={}, xnat_instance=intf)
        
        print('{} tests (`{}`):'.format(spmv.__class__.__name__, spmv.version))
        spmv.tests
        ```
            SPM12SegmentValidator tests (`d6ca22c1`):
            
            [<bbrc.validation.processing.spm.HasCorrectNumberOfItems at 0x273dee24e88>,
             <bbrc.validation.processing.spm.HasCorrectItems at 0x273dee247c8>,
             <bbrc.validation.processing.spm.HasCorrectSPMVersion at 0x273dda8f4c8>,
             <bbrc.validation.processing.spm.HasCorrectMatlabVersion at 0x273dee28848>,
             <bbrc.validation.processing.spm.HasCorrectOSVersion at 0x273dee287c8>,
             <bbrc.validation.processing.spm.SPM12SegmentSnapshot at 0x273dee249c8>,
             <bbrc.validation.processing.spm.HasNormalSPM12Volumes at 0x273dee28788>,
             <bbrc.validation.processing.spm.SPM12SegmentExecutionTime at 0x273dee28bc8>]
        
        #### Run `SPM12SegmentValidator` against an MRI session,  
        ```python
        spmv.run('XNAT_E00001')
        ```
        
            2021-02-04 12:12:54,635 - root - INFO - Running <bbrc.validation.processing.spm.HasCorrectNumberOfItems object at 0x00000273DEE24E88>
            2021-02-04 12:12:54,964 - root - ERROR - XNAT_E00001 has 15 items (different from 16)
            2021-02-04 12:12:55,572 - root - INFO - Running <bbrc.validation.processing.spm.HasCorrectItems object at 0x00000273DEE247C8>
            2021-02-04 12:12:56,120 - root - INFO - Running <bbrc.validation.processing.spm.HasCorrectSPMVersion object at 0x00000273DDA8F4C8>
            2021-02-04 12:12:56,592 - root - INFO - Running <bbrc.validation.processing.spm.HasCorrectMatlabVersion object at 0x00000273DEE28848>
            2021-02-04 12:12:56,782 - root - INFO - Running <bbrc.validation.processing.spm.HasCorrectOSVersion object at 0x00000273DEE287C8>
            2021-02-04 12:12:57,001 - root - INFO - Running <bbrc.validation.processing.spm.SPM12SegmentSnapshot object at 0x00000273DEE249C8>
            2021-02-04 12:13:04,997 - root - INFO - * Creating snapshots...
            2021-02-04 12:13:46,472 - root - INFO - Saved in /tmp/tmp3j664u27.png
            2021-02-04 12:13:46,515 - root - INFO - Running <bbrc.validation.processing.spm.HasNormalSPM12Volumes object at 0x00000273DEE28788>
            2021-02-04 12:13:50,552 - root - INFO - Running <bbrc.validation.processing.spm.SPM12SegmentExecutionTime object at 0x00000273DEE28BC8>
        
        #### Collect results from `SPM12SegmentValidator` execution,
        ```python
        import json 
        result = spmv.dump()
        json.loads(result)
        ```
            {'experiment_id': 'XNAT_E00001',
             'version': 'd6ca22c1',
             'generated': '2021-02-04, 12:13',
             'HasCorrectItems': {'has_passed': False,
              'data': ["Missing SPM12_SEGMENT items: ['pyscript_setorigin.m']."]},
             'HasCorrectSPMVersion': {'has_passed': True, 'data': []},
             'HasCorrectMatlabVersion': {'has_passed': True, 'data': []},
             'HasCorrectOSVersion': {'has_passed': True, 'data': []},
             'SPM12SegmentSnapshot': {'has_passed': True, 
              'data': ['/tmp/tmp3j664u27.png']},
             'HasNormalSPM12Volumes': {'has_passed': True,
              'data': ['Volumes: 773592.1702940931 524339.7480925963']},
             'SPM12SegmentExecutionTime': {'has_passed': True, 'data': ['0:07:15']}}
        
        #### Generate a human-readable PDF report from the results,
        ```python
        import tempfile
        _,fp = tempfile.mkstemp(suffix='.pdf')
        spmv.report(fp)
        print('Report created: {}'.format(fp))
        
        ```
            Loading pages (1/6)
            Counting pages (2/6)                                               
            Resolving links (4/6)                                                       
            Loading headers and footers (5/6)                                           
            Printing pages (6/6)
            Done                                                                      
            
            Report created: '/home/jhuguet/notebooks/bbrc-validator/tmpcexwvwj5.pdf'
        
        Install
        -------
        
        **bbrc-validator** can be installed via `pip`,
        ```bash
        pip install bbrc-validator
        ```
        
        `bbrc-validator` requires [wkhtmltopdf](http://wkhtmltopdf.org/) for PDF 
        generation. A static build release (with QT patches) is recommended, see 
        available releases 
        [here](https://wkhtmltopdf.org/downloads.html) by OS/distribution.
        
        On Ubuntu 18.04:
        ```bash
        wget https://github.com/wkhtmltopdf/packaging/releases/download/0.12.6-1/wkhtmltox_0.12.6-1.bionic_amd64.deb
        dpkg -i wkhtmltox_0.12.6-1.bionic_amd64.deb
        apt --fix-broken -y install
        ```
        On CentOS 7:
        ```bash
        wget https://github.com/wkhtmltopdf/packaging/releases/download/0.12.6-1/wkhtmltox-0.12.6-1.centos7.x86_64.rpm
        yum -y  localinstall wkhtmltox-0.12.6-1.centos7.x86_64.rpm
        ```
        
        Contributing
        ------------
        
        **bbrc-validator** is still under active development. The currently included _Tests_ 
        and _Validators_ have been tailored to the particular needs and context of the 
        Barcelonaβeta Brain Research Center and as such might differ with the needs from 
        other organizations.  
        However, the software was designed with an aim towards genericity, modularity and 
        reusability. Since all Tests are based upon the same template (eg. each of them 
        being linked to XNAT data resources as test cases), this makes them virtually 
        shareable across groups and makes **bbrc-validator** open to public contributions.
        
        Contact us for details on how to contribute or open an issue to start a discussion.
        
        [![BBRC](https://www.barcelonabeta.org/sites/default/files/logo-barcelona-beta_0.png)
        ](https://barcelonabeta.org/)
Platform: UNKNOWN
Classifier: Intended Audience :: Science/Research
Classifier: Intended Audience :: Developers
Classifier: Topic :: Scientific/Engineering
Classifier: Operating System :: Unix
Classifier: Programming Language :: Python :: 3.7
Description-Content-Type: text/markdown
