Mounting and object references

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Mounting and object references

Thomas Krijnen
Dear all,

What are the possibilities for creating object references to and from mounted files? To me this is a very powerful scenario where you can for example provide datasets in varying levels of detail and by mounting, select the configuration to your liking. My initial tests where encouraging, but my premonition now is that these initial successful attempts where just based on object headers aligning by coincidence and that this is in fact not supported.

It seems that what is written to the hobj_ref_t is not taking into account any information pertaining to the mount points, in spite of the fact that the function takes a full path as an argument by which it would traverse file boundaries.

See below for [1] a C++ example where I try to accomplish this and [2] a diagram of what I thought to accomplish. Note that the

What are my options here (including possible work arounds)? Thanks in advance.

Kind regards,
Thomas

[1]
//~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
#include <H5Cpp.h>

#include <iostream>

void write() {
    H5::H5File f("test.hdf", H5F_ACC_TRUNC);
    H5::H5File child1("child1.hdf", H5F_ACC_TRUNC);
   
    hsize_t dims = 2;
    H5::DataSpace space(1, &dims);

    H5::Group target = f.createGroup("target");

    H5::DataSet dataset1 = child1.createDataSet("dataset1", H5::PredType::STD_REF_OBJ, space);
    H5::Group target2 = child1.createGroup("target2");
   
    f.createGroup("child1").close();
    f.mount("child1", child1, H5::PropList::DEFAULT);

    hobj_ref_t data[2];
    f.reference(&data[0], "/target");
    f.reference(&data[1], "/child1/target2");

    dataset1.write(data, H5::PredType::STD_REF_OBJ);
}

void read() {
    H5::H5File f("test.hdf", H5F_ACC_RDONLY);
    H5::H5File child1("child1.hdf", H5F_ACC_RDONLY);

    f.mount("child1", child1, H5::PropList::DEFAULT);

    hobj_ref_t data[2];
    H5::DataSet dataset1 = f.openDataSet("/child1/dataset1");
    dataset1.read(data, H5::PredType::STD_REF_OBJ);
   
    for (int i = 0; i < 2; ++i) {
        H5::Group target;
        target.dereference(f, &data[i], H5R_OBJECT);
        std::cout << target.getObjName() << std::endl;
    }
    // note that target.dereference(child1, &data[1], H5R_OBJECT); would work but only if not previously mounted
}

int main(int, char**) {
    write();
    read();

    return 0;
}
//~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

[2]
test.hdf
  \
   +-target   <--------\
   |                   |
   +-------child1.hdf--|---+
   |\                  |   |
   | +-dataset2  +-----|-+ |
   | |           |{0}-/  | |
   | |           |{1}-\  | |
   | |           +-----|-+ |
   | |                 |   |
   | |                 |   |
   | |                 |   |
   | +-target2  <------/   |
   |                       |
   +-----------------------+

_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
Reply | Threaded
Open this post in threaded view
|

Re: Mounting and object references

Elena Pourmal
Hi Thomas,

You are correct. HDF5 currently supports object and region references to the objects stored in the same HDF5 file.

It would be nice to expand the feature, but at this point it is not on our to-do list. Contributions to the code will be definitely considered!

Thank you!

Elena

On Nov 22, 2017, at 11:13 AM, Thomas Krijnen <[hidden email]> wrote:

Dear all,

What are the possibilities for creating object references to and from mounted files? To me this is a very powerful scenario where you can for example provide datasets in varying levels of detail and by mounting, select the configuration to your liking. My initial tests where encouraging, but my premonition now is that these initial successful attempts where just based on object headers aligning by coincidence and that this is in fact not supported.

It seems that what is written to the hobj_ref_t is not taking into account any information pertaining to the mount points, in spite of the fact that the function takes a full path as an argument by which it would traverse file boundaries.

See below for [1] a C++ example where I try to accomplish this and [2] a diagram of what I thought to accomplish. Note that the

What are my options here (including possible work arounds)? Thanks in advance.

Kind regards,
Thomas

[1]
//~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
#include <H5Cpp.h>

#include <iostream>

void write() {
    H5::H5File f("test.hdf", H5F_ACC_TRUNC);
    H5::H5File child1("child1.hdf", H5F_ACC_TRUNC);
   
    hsize_t dims = 2;
    H5::DataSpace space(1, &dims);

    H5::Group target = f.createGroup("target");

    H5::DataSet dataset1 = child1.createDataSet("dataset1", H5::PredType::STD_REF_OBJ, space);
    H5::Group target2 = child1.createGroup("target2");
   
    f.createGroup("child1").close();
    f.mount("child1", child1, H5::PropList::DEFAULT);

    hobj_ref_t data[2];
    f.reference(&data[0], "/target");
    f.reference(&data[1], "/child1/target2");

    dataset1.write(data, H5::PredType::STD_REF_OBJ);
}

void read() {
    H5::H5File f("test.hdf", H5F_ACC_RDONLY);
    H5::H5File child1("child1.hdf", H5F_ACC_RDONLY);

    f.mount("child1", child1, H5::PropList::DEFAULT);

    hobj_ref_t data[2];
    H5::DataSet dataset1 = f.openDataSet("/child1/dataset1");
    dataset1.read(data, H5::PredType::STD_REF_OBJ);
   
    for (int i = 0; i < 2; ++i) {
        H5::Group target;
        target.dereference(f, &data[i], H5R_OBJECT);
        std::cout << target.getObjName() << std::endl;
    }
    // note that target.dereference(child1, &data[1], H5R_OBJECT); would work but only if not previously mounted
}

int main(int, char**) {
    write();
    read();

    return 0;
}
//~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

[2]
test.hdf
  \
   +-target   <--------\
   |                   |
   +-------child1.hdf--|---+
   |\                  |   |
   | +-dataset2  +-----|-+ |
   | |           |{0}-/  | |
   | |           |{1}-\  | |
   | |           +-----|-+ |
   | |                 |   |
   | |                 |   |
   | |                 |   |
   | +-target2  <------/   |
   |                       |
   +-----------------------+
_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5


_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5