Loading and storing IMAS data¶

IMAS data is grouped together in Data Entries. A Data Entry is a collection of IDSs and their (potentially) multiple occurrences, which groups and stores data over multiple IDSs as a single dataset. The Data Entry concept is used whether the collection of IDSs is stored in a database or only exists temporarily (for example for communication in an integrated workflow).

Loading and storing IMAS data happens through an IMAS Database Entry. A Database Entry tracks the information required for locating where the Data Entry is (or will be) stored on disk. In C++ this object is modeled as the IdsNs::IDS class.

You may Open an existing IMAS Database Entry, which you can use for loading data that was stored previously. Alternatively you can Create a new IMAS Database Entry to store IDS data.

Open an existing IMAS Database Entry¶

To open an IMAS Database Entry, you need to know the URI indicating where the Access Layer can find the data. IMAS URIs start with imas: and indicate the format and the location of the stored data. You can find a detailed description of the IMAS URI syntax on the IMAS Data Entry URIs page.

// Include the Access Layer
#include "ALClasses.h"
#include <iostream>

int main(int argc, char *argv[]) {
    // Create a new IDS object
    IdsNs::IDS ids;
    // Open the database entry by providing an IMAS URI
    ids.open("imas:mdsplus?user=public;pulse=131024;run=41;database=ITER;version=3", OPEN_PULSE);

    // Alternatively, we can open an IDS providing legacy
    // arguments: (pulse, run, 0, 0)
    IdsNs::IDS legacy(131024, 41, 0, 0);
    // And open with the openEnv method
    legacy.openEnv("public", "ITER", "3");

    // Use setBackend to use a different backend than MDS+
    IdsNs::IDS legacy2(1, 1, 0, 0);
    legacy2.setBackend(MEMORY_BACKEND);
    legacy.openEnv("public", "ITER", "3");

    return 0;
}

Loading IMAS data¶

After you open a database entry, you can request to load data from disk.

Load an entire IDS¶

With IdsNs::Ids::get() you can load (“get”) an entire IDS from the database entry.

Multiple occurrences of an IDS may be stored in a data entry. By default, if you don’t specify an occurrence number, occurrence 0 is loaded. By providing an occurrence number you can load a specific occurrence. How different occurrences are used depends on the experiment. They could, for example, correspond to:

• different methods for computing the physical quantities of the IDS, or

• different functionalities in a workflow (e.g. initial values, prescribed values, values at next time step, …), or

• multiple subsystems (e.g. diagnostics) of the same type in an experiment, etc.

// Include the Access Layer
#include "ALClasses.h"
#include <iostream>

int main(int argc, char *argv[]) {
    // Create a new IDS object
    IdsNs::IDS ids;
    // Open the database entry by providing an IMAS URI
    ids.open("imas:mdsplus?user=public;pulse=131024;run=41;database=ITER;version=3", OPEN_PULSE);

    // Load the core_profiles IDS, occurrence 0
    ids._core_profiles.get();
    // Load the core_profiles IDS, occurrence 1, beware
    // that this overwrites the contents of occurrence 0!
    ids._core_profiles.get(1);

    return 0;
}

Load a single time slice of an IDS¶

Instead of loading a full IDS from disk, the Access Layer allows you to load a specific time slice. This is often useful when you’re not interested in the full time evolution, but instead want data of a specific time. You can use IdsNs::Ids::getSlice() for this.

Most of the time there are no entries at that specific time, so you also need to indicate an interpolation method. This determines what values the access layer returns when your requested time is in between available time points in the data. Three interpolation methods currently exist:

PREVIOUS_SAMPLE

Returns the previous time slice if the requested time does not exactly exist in the original IDS.

For example, when data exists at \(t=\{1, 3, 4\}\), requesting \(t_r=2.1\) will give you the data at \(t=1\).

Edge case behaviour. \(\{t_i\}, i=1..N\) represents the time series stored in the IDS.¶

Case	Behaviour
\(t_r \lt t_1\)	Return data at \(t_1\).
\(t_r = t_i\) [1]	Return data at \(t_i\).

CLOSEST_SAMPLE

Returns the closest time slice in the original IDS. This can also be after the requested time.

For example, when data exists at \(t=\{1, 3, 4\}\), requesting \(t=2.1\) will give you the data at \(t=3\).

Edge case behaviour. \(\{t_i\}, i=1..N\) represents the time series stored in the IDS.¶

Case	Behaviour
\(t_r \lt t_1\)	Return data at \(t_1\).
\(t_r = t_i\) [1]	Return data at \(t_i\).
\(t_r - t_i = t_{i+1} - t_r\) [1]	Return data at \(t_{i+1}\).

INTERPOLATION

Returns a linear interpolation between the existing slices before and after the requested time.

For example, when data exists at \(t=\{1, 3, 4\}\), requesting \(t=2.1\) will give you a linear interpolation of the data at \(t=1\) and the data at \(t=3\).

Note that the linear interpolation will be successful only if, between the two time slices of an interpolated dynamic array of structure, the same leaves are populated and they have the same size. Otherwise IdsNs::Ids::getSlice() will interpolate all fields with a compatible size and leave others empty.

Edge case behaviour. \(\{t_i\}, i=1..N\) represents the time series stored in the IDS.¶

Case	Behaviour
\(t_r \lt t_1\)	Return data at \(t_1\).
\(t_r \gt t_N\)	Return data at \(t_N\).

// Include the Access Layer
#include "ALClasses.h"
#include <iostream>

int main(int argc, char *argv[]) {
    // Open the database entry
    IdsNs::IDS ids;
    ids.open("imas:mdsplus?user=public;pulse=131024;run=41;database=ITER;version=3", OPEN_PULSE);

    // Retrieve the time slice just before t=370 of the core_profiles IDS
    ids._core_profiles.getSlice(370, PREVIOUS_SAMPLE);

    return 0;
}

Create a new IMAS Database Entry¶

To create a new IMAS Database Entry, you need to provide the URI to indicate the format and the location where you want to store the data. You can find a detailed description of the IMAS URI syntax and the options available on the IMAS Data Entry URIs page.

// Include the Access Layer
#include "ALClasses.h"
#include <iostream>

int main(int argc, char *argv[]) {
    // Create a new IDS object
    IdsNs::IDS ids;
    // Create the database entry by providing an IMAS URI
    ids.open("imas:hdf5?path=my-data", FORCE_CREATE_PULSE);

    return 0;
}

Store IMAS data¶

After you have created an IMAS Database Entry, you can use it for storing IDS data. There are two ways to do this:

Store an entire IDS¶

With IdsNs::Ids::put() you can store (“put”) an entire IDS in a database entry. First you need to have an IDS with data: you can create a new one or load an IDS which you modify. See Use Interface Data Structures for more information on using and manipulating IDSs.

Multiple occurrences of an IDS may be stored in a data entry. By default, if you don’t specify an occurrence number, the IDS is stored as occurrence 0. By providing an occurrence number you can store the IDS as a specific occurrence.

// Include the Access Layer
#include "ALClasses.h"
#include <iostream>

int main(int argc, char *argv[]) {
    // Create the database entry
    IdsNs::IDS ids;
    ids.open("imas:hdf5?path=my-data", FORCE_CREATE_PULSE);

    // Create an empty pf_active IDS
    IdsNs::IDS::pf_active pf_active;
    // Set the mandatory ids_properties.homogeneous_time field
    pf_active.ids_properties.homogeneous_time = IDS_TIME_MODE_HOMOGENEOUS;
    // Continue filling the pf_active IDS here
    // ...

    // Associate the pf_active IDS with our data entry
    pf_active.setPulseCtx(ids.getPulseCtx());
    // And store it
    pf_active.put();

    // Alternatively, store the pf_active IDS as occurrence 1
    pf_active.put(1);

    return 0;
}

Append a time slice to an already-stored IDS¶

With IdsNs::Ids::putSlice() you can append a time slice to an existing database entry. This is useful when you generate data inside a time loop (for example in simulations, or when taking measurements of an experiment).

It means you can put a time slice with every iteration of your loop such that you don’t have to keep track of the complete time evolution in memory. Instead, the Access Layer will keep appending the data to the Database Entry in the storage backend.

// Include the Access Layer
#include "ALClasses.h"
#include <iostream>

int main(int argc, char *argv[]) {
    // Create the database entry
    IdsNs::IDS ids;
    ids.open("imas:hdf5?path=my-data", FORCE_CREATE_PULSE);

    // Create an empty core_profiles IDS
    IdsNs::IDS::core_profiles core_profiles;
    // Set the mandatory ids_properties.homogeneous_time field
    core_profiles.ids_properties.homogeneous_time = IDS_TIME_MODE_HOMOGENEOUS;
    // Continue filling the core_profiles IDS with static values here
    // ...

    // Associate the core_profiles IDS with our data entry
    core_profiles.setPulseCtx(ids.getPulseCtx());
    // Allocate time array
    core_profiles.time.resize(1);

    // Example time integration loop:
    double t = 0.0;
    double t_stop = 60.0;
    double dt = 0.1;
    while (t < t_stop)
    {
        core_profiles.time = t;
        // Fill your time-dependent data in the core_profiles IDS here
        // ...

        // Append the current time slice to the data stored on disk
        core_profiles.putSlice();
        t += dt;
    }

    return 0;
}

Listing all occurrences of an IDS from a backend¶

With IdsNs::Ids::list_all_occurrences() you can List all non-empty occurrences of an IDS using its name in the dataset, and optionnally return the content of a descriptive node path.

// Include the Access Layer
#include "ALClasses.h"
#include <iostream>

int main(int argc, char *argv[]) {
    // Create a new IDS object
    IdsNs::IDS ids;
    std::vector<string> node_content_list;
    std::vector<int> occurrence_list;
    int status;
    // Open the database entry by providing an IMAS URI
    ids.open("imas:mdsplus?user=public;pulse=131024;run=41;database=ITER;version=3", OPEN_PULSE);

    // get all occurrence of all "ids_properties/comment" of each 'distributions' IDS
    status = IdsNs::IDS::list_all_occurrences(ids.getPulseCtx(), "distributions", "ids_properties/comment", node_content_list, occurrence_list);
    if (status==0)
        for (int i = 0; i < node_content_list.size(); i ++)
            std::cout << i << " " << occurrence_list[i] << " " << node_content_list[i] << std::endl;
    return 0;
}