Skip to content

Running an experiment

run_fedpydeseq2_experiment(n_centers=2, backend='subprocess', register_data=False, simulate=True, asset_directory=None, centers_root_directory=None, compute_plan_name='FedPyDESeq2Experiment', dataset_name='MyDatasetName', remote_timeout=86400, clean_models=True, save_filepath=None, credentials_path=None, dataset_datasamples_keys_path=None, cp_id_path=None, parameter_file=None, fedpydeseq2_wheel_path=None, **kwargs)

Run a federated experiment using the DESeq2 strategy.

Parameters:

Name Type Description Default
n_centers int

Number of centers to use in the federated experiment.

 2
backend BackendType

Backend to use for the experiment. Should be one of "subprocess", "docker" or "remote".

 'subprocess'
register_data bool

Whether to register the data on the substra platform. Can be True only when using the remote backend.

 False
simulate bool

Whether to simulate the experiment. If True, the experiment will be simulated and no data will be sent to the centers. This can be True only in subprocess backend.

 True
asset_directory Path

Path to the directory containing the assets (opener.py and description.md).

 None
centers_root_directory Path

Path to the directory containing the centers data. Can be None only in remote mode when register_data is False. The centers data should be organized as follows: 

<centers_root_directory>
├── center_0
│   ├── counts_data.csv
│   └── metadata.csv
├── center_1
│   ├── counts_data.csv
│   └── metadata.csv
└──
where the metadata.csv file is indexed by sample barcodes and contains all columns needed to build the design matrix, and the counts_data.csv file represents a dataframe with gene names as columns and sample barcodes as rows, in the "barcode" column.

 None
compute_plan_name str

Name of the compute plan to use for the experiment.

 'FedPyDESeq2Experiment'
dataset_name str

Name of the dataset to fill in the Dataset schema.

 'MyDatasetName'
remote_timeout int

Timeout in seconds for the remote backend.

 86400
clean_models bool

Whether to clean the models after the experiment.

 True
save_filepath str or Path

Path to save the results of the experiment.

 None
credentials_path str or Path

Path to the file containing the credentials to use for the remote backend.

 None
dataset_datasamples_keys_path str or Path

Path to the file containing the datasamples keys of the dataset. Only used for the remote backend. Is filled in if register_data is True, and read if register_data is False.

 None
cp_id_path str or Path

Path to the file containing the compute plan id. This file is a yaml file with the following structure: 

algo_org_name: str
credentials_path: str
compute_plan_key: str

None
parameter_file str or Path

If not None, yaml file containing the parameters to pass to the DESeq2Strategy. If None, the default parameters are used.

 None
fedpydeseq2_wheel_path str or Path

Path to the wheel file of the fedpydeseq2 package. If provided and the backend is remote, this wheel will be added to the dependencies.

 None
**kwargs

Arguments to pass to the DESeq2Strategy. They will overwrite those specified in the parameter_file if the file is not None.

 {}

Returns:

Type Description
dict

Result of the strategy, which are assumed to be contained in the results attribute of the last round of the aggregation node.
Source code in fedpydeseq2/fedpydeseq2_pipeline.py 
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
def run_fedpydeseq2_experiment(
    n_centers: int = 2,
    backend: BackendType = "subprocess",
    register_data: bool = False,
    simulate: bool = True,
    asset_directory: Path | None = None,
    centers_root_directory: Path | None = None,
    compute_plan_name: str = "FedPyDESeq2Experiment",
    dataset_name: str = "MyDatasetName",
    remote_timeout: int = 86400,  # 24 hours
    clean_models: bool = True,
    save_filepath: str | Path | None = None,
    credentials_path: str | Path | None = None,
    dataset_datasamples_keys_path: str | Path | None = None,
    cp_id_path: str | Path | None = None,
    parameter_file: str | Path | None = None,
    fedpydeseq2_wheel_path: str | Path | None = None,
    **kwargs,
) -> dict:
    """Run a federated experiment using the DESeq2 strategy.

    Parameters
    ----------
    n_centers : int
        Number of centers to use in the federated experiment.

    backend : BackendType
        Backend to use for the experiment. Should be one of "subprocess", "docker"
        or "remote".

    register_data : bool
        Whether to register the data on the substra platform. Can be True only
        when using the remote backend.

    simulate : bool
        Whether to simulate the experiment. If True, the experiment will be simulated
        and no data will be sent to the centers. This can be True only in subprocess
        backend.

    asset_directory : Path
        Path to the directory containing the assets (opener.py and description.md).

    centers_root_directory : Path, optional
        Path to the directory containing the centers data. Can be None only in remote
        mode when register_data is False.
        The centers data should be organized as follows:
        ```
        <centers_root_directory>
        ├── center_0
        │   ├── counts_data.csv
        │   └── metadata.csv
        ├── center_1
        │   ├── counts_data.csv
        │   └── metadata.csv
        └──

        ```
        where the metadata.csv file is indexed by sample barcodes and contains
        all columns needed to build the design matrix, and the counts_data.csv file
        represents a dataframe with gene names as columns and sample barcodes as rows,
        in the "barcode" column.

    compute_plan_name : str
        Name of the compute plan to use for the experiment.

    dataset_name : str
        Name of the dataset to fill in the Dataset schema.

    remote_timeout : int
        Timeout in seconds for the remote backend.

    clean_models : bool
        Whether to clean the models after the experiment.

    save_filepath : str or Path
        Path to save the results of the experiment.

    credentials_path : str or Path
        Path to the file containing the credentials to use for the remote backend.

    dataset_datasamples_keys_path : str or Path
        Path to the file containing the datasamples keys of the dataset.
        Only used for the remote backend.
        Is filled in if register_data is True, and read if register_data is False.

    cp_id_path : str or Path, optional
        Path to the file containing the compute plan id.
        This file is a yaml file with the following structure:
        ```
        algo_org_name: str
        credentials_path: str
        compute_plan_key: str
        ```

    parameter_file : str or Path, optional
        If not None, yaml file containing the parameters to pass to the DESeq2Strategy.
        If None, the default parameters are used.

    fedpydeseq2_wheel_path : str or Path, optional
        Path to the wheel file of the fedpydeseq2 package. If provided and the backend
        is remote, this wheel will be added to the dependencies.

    **kwargs
        Arguments to pass to the DESeq2Strategy. They will overwrite those specified
        in the parameter_file if the file is not None.

    Returns
    -------
    dict
        Result of the strategy, which are assumed to be contained in the
        results attribute of the last round of the aggregation node.


    """
    if parameter_file is not None:
        with open(parameter_file, "rb") as file:
            parameters = yaml.load(file, Loader=yaml.FullLoader)
    else:
        parameters = {}
    parameters.update(kwargs)
    strategy = DESeq2Strategy(**parameters)

    return run_federated_experiment(
        strategy=strategy,
        n_centers=n_centers,
        backend=backend,
        register_data=register_data,
        simulate=simulate,
        centers_root_directory=centers_root_directory,
        assets_directory=asset_directory,
        compute_plan_name=compute_plan_name,
        dataset_name=dataset_name,
        remote_timeout=remote_timeout,
        clean_models=clean_models,
        save_filepath=save_filepath,
        credentials_path=credentials_path,
        dataset_datasamples_keys_path=dataset_datasamples_keys_path,
        cp_id_path=cp_id_path,
        fedpydeseq2_wheel_path=fedpydeseq2_wheel_path,
    )