Reading Nested Compound objects in C++

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Reading Nested Compound objects in C++

Todd Dobmeyer
All

I have an HDF5 file I need to read in a C++ application. I performed an h5dump on the data and have the following structure (with many items removed that I am not interested in). I have successfully read in all 7500 values of "num_samps" using the C++ HDF5 library. What I am struggling to figure out how to read is the array of compound "iq_data". I know in the datatype it says there are 1000 compound objects. I don't know if I can get this value out of the DataType, but I do know the num_samps points to this array size as well, which is fine since I can read this. Can any of you help me figure out how to read the "iq_data" array. Below is the sample data followed by my code I have so far. In my code "numSamps" is good, but "iqData" is not holding anything useful. I am sure I am missing something.

Thanks for your help!
Todd Dobmeyer

--------------SAMPLE DATA-------------------------

HDF5 "hdfFile.be01" {
GROUP "/" {
   DATATYPE "DRdata" H5T_COMPOUND {
      H5T_ARRAY { [5] H5T_STD_U32LE } "guid";
      H5T_STD_I32LE "version";
      ...
      H5T_STD_U32LE "num_samps";
      ...
      H5T_STD_U32LE "sequence_dummy";
      H5T_ARRAY { [1000] H5T_COMPOUND {
         H5T_IEEE_F32LE "f_i";
         H5T_IEEE_F32LE "f_q";
      } } "iq_data";
   }
   DATASET "DRx1data" {
      DATATYPE  "/DRx1"
      DATASPACE  SIMPLE { ( 7500 ) / ( H5S_UNLIMITED ) }
      DATA {
      (0): {
            [ 352430272, 25602, 2314, 1442149219, 43629 ],
            1,
            1.44215e+09,
            1442149219,
            0,
            1,
            1,
            0,
            1,
            0,
            0,
            0,
            3,
            0,
            0,
            0,
            0,
            1,
            31095000,
            20000,
            0,
            1000,
            1000, // (this is num_samps)
            1,
            1000,
            [ {
                  1.09068e+09,
                  4.53469e+08
               }, {
                  4.03303e+08,
                  4.11846e+08
               }, {
                  -1.35295e+08,
                  -1.70107e+08
               }, ...

--------------------CODE------------------


typedef struct {
    float f_i;
    float f_q;
} complex_t;

int main(int argc, char *argv[])
{
    std::string path = "/data/hdfFile.be01";
    // Open the file, dataset, and dataspace
    H5::H5File file(path, H5F_ACC_RDONLY);
    H5::DataSet dataset = file.openDataSet("DRx1data");
    H5::DataSpace dataspace = dataset.getSpace();
    
    // Get the number of elements and number of samples
    int nelems = dataspace.getSimpleExtentNpoints();
    int numSamps[nelems];
    
    // Create number of samples type
    H5::CompType sampsType(sizeof(uint));
    sampsType.insertMember("num_samps", 0, H5::PredType::STD_U32LE);
    
    // Read the number of samples
    dataset.read(numSamps, sampsType);
    
    // Now try to read the I/Q data
    H5::CompType iqDataType(sizeof(complex_t));
    iqDataType.insertMember("f_i", HOFFSET(complex_t, f_i), H5::PredType::NATIVE_FLOAT);
    iqDataType.insertMember("f_q", HOFFSET(complex_t, f_q), H5::PredType::NATIVE_FLOAT);
    complex_t** iqData = new complex_t*[nelems];
    for(int i = 0 ; i < nelems ; ++i)
    {
        iqData[i] = new complex_t[numSamps[i]];
    }
    dataset.read(iqData, iqDataType);

_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Reading Nested Compound objects in C++

Binh-Minh Ribler

Hi Todd,


I'm not sure I understood your data completely, but I wonder what you will get if you change this line:


complex_t** iqData = new complex_t*[nelems];

to:


complex_t* iqData = new complex_t[nelems];

and read without the for loop...


Binh-Minh



From: Hdf-forum <[hidden email]> on behalf of Todd Dobmeyer <[hidden email]>
Sent: Monday, July 11, 2016 3:10 PM
To: [hidden email]
Subject: [Hdf-forum] Reading Nested Compound objects in C++
 
All

I have an HDF5 file I need to read in a C++ application. I performed an h5dump on the data and have the following structure (with many items removed that I am not interested in). I have successfully read in all 7500 values of "num_samps" using the C++ HDF5 library. What I am struggling to figure out how to read is the array of compound "iq_data". I know in the datatype it says there are 1000 compound objects. I don't know if I can get this value out of the DataType, but I do know the num_samps points to this array size as well, which is fine since I can read this. Can any of you help me figure out how to read the "iq_data" array. Below is the sample data followed by my code I have so far. In my code "numSamps" is good, but "iqData" is not holding anything useful. I am sure I am missing something.

Thanks for your help!
Todd Dobmeyer

--------------SAMPLE DATA-------------------------

HDF5 "hdfFile.be01" {
GROUP "/" {
   DATATYPE "DRdata" H5T_COMPOUND {
      H5T_ARRAY { [5] H5T_STD_U32LE } "guid";
      H5T_STD_I32LE "version";
      ...
      H5T_STD_U32LE "num_samps";
      ...
      H5T_STD_U32LE "sequence_dummy";
      H5T_ARRAY { [1000] H5T_COMPOUND {
         H5T_IEEE_F32LE "f_i";
         H5T_IEEE_F32LE "f_q";
      } } "iq_data";
   }
   DATASET "DRx1data" {
      DATATYPE  "/DRx1"
      DATASPACE  SIMPLE { ( 7500 ) / ( H5S_UNLIMITED ) }
      DATA {
      (0): {
            [ 352430272, 25602, 2314, 1442149219, 43629 ],
            1,
            1.44215e+09,
            1442149219,
            0,
            1,
            1,
            0,
            1,
            0,
            0,
            0,
            3,
            0,
            0,
            0,
            0,
            1,
            31095000,
            20000,
            0,
            1000,
            1000, // (this is num_samps)
            1,
            1000,
            [ {
                  1.09068e+09,
                  4.53469e+08
               }, {
                  4.03303e+08,
                  4.11846e+08
               }, {
                  -1.35295e+08,
                  -1.70107e+08
               }, ...

--------------------CODE------------------


typedef struct {
    float f_i;
    float f_q;
} complex_t;

int main(int argc, char *argv[])
{
    std::string path = "/data/hdfFile.be01";
    // Open the file, dataset, and dataspace
    H5::H5File file(path, H5F_ACC_RDONLY);
    H5::DataSet dataset = file.openDataSet("DRx1data");
    H5::DataSpace dataspace = dataset.getSpace();
    
    // Get the number of elements and number of samples
    int nelems = dataspace.getSimpleExtentNpoints();
    int numSamps[nelems];
    
    // Create number of samples type
    H5::CompType sampsType(sizeof(uint));
    sampsType.insertMember("num_samps", 0, H5::PredType::STD_U32LE);
    
    // Read the number of samples
    dataset.read(numSamps, sampsType);
    
    // Now try to read the I/Q data
    H5::CompType iqDataType(sizeof(complex_t));
    iqDataType.insertMember("f_i", HOFFSET(complex_t, f_i), H5::PredType::NATIVE_FLOAT);
    iqDataType.insertMember("f_q", HOFFSET(complex_t, f_q), H5::PredType::NATIVE_FLOAT);
    complex_t** iqData = new complex_t*[nelems];
    for(int i = 0 ; i < nelems ; ++i)
    {
        iqData[i] = new complex_t[numSamps[i]];
    }
    dataset.read(iqData, iqDataType);


_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Reading Nested Compound objects in C++

Todd Dobmeyer
Binh-Minh

Making your change keeps me from crashing, but unfortunately doesn't get any of the data.  So to better understand my data, I have 7500 pieces of DATA in the DATASPACE. Inside 1 piece of data is an array of Compound objects each containing 2 floats. The Compound object is called iq_data and the 2 floats inside are f_i and f_q.

I originally did the 2D array thinking I needed the outside dimension to be 7500 and the inner dimensions to be 1000. But now I think I need a 1D array that holds 7500*1000 compound objects or 7500*1000*2 floats. However, I cannot figure out how to read the data. Do I read "iq_data" or do I try and read "f_i" and "f_q"?

Thanks again and I hope I clarified things!
Todd

On Tue, Jul 12, 2016 at 2:14 AM, Binh-Minh Ribler <[hidden email]> wrote:

Hi Todd,


I'm not sure I understood your data completely, but I wonder what you will get if you change this line:


complex_t** iqData = new complex_t*[nelems];

to:


complex_t* iqData = new complex_t[nelems];

and read without the for loop...


Binh-Minh



From: Hdf-forum <[hidden email]> on behalf of Todd Dobmeyer <[hidden email]>
Sent: Monday, July 11, 2016 3:10 PM
To: [hidden email]
Subject: [Hdf-forum] Reading Nested Compound objects in C++
 
All

I have an HDF5 file I need to read in a C++ application. I performed an h5dump on the data and have the following structure (with many items removed that I am not interested in). I have successfully read in all 7500 values of "num_samps" using the C++ HDF5 library. What I am struggling to figure out how to read is the array of compound "iq_data". I know in the datatype it says there are 1000 compound objects. I don't know if I can get this value out of the DataType, but I do know the num_samps points to this array size as well, which is fine since I can read this. Can any of you help me figure out how to read the "iq_data" array. Below is the sample data followed by my code I have so far. In my code "numSamps" is good, but "iqData" is not holding anything useful. I am sure I am missing something.

Thanks for your help!
Todd Dobmeyer

--------------SAMPLE DATA-------------------------

HDF5 "hdfFile.be01" {
GROUP "/" {
   DATATYPE "DRdata" H5T_COMPOUND {
      H5T_ARRAY { [5] H5T_STD_U32LE } "guid";
      H5T_STD_I32LE "version";
      ...
      H5T_STD_U32LE "num_samps";
      ...
      H5T_STD_U32LE "sequence_dummy";
      H5T_ARRAY { [1000] H5T_COMPOUND {
         H5T_IEEE_F32LE "f_i";
         H5T_IEEE_F32LE "f_q";
      } } "iq_data";
   }
   DATASET "DRx1data" {
      DATATYPE  "/DRx1"
      DATASPACE  SIMPLE { ( 7500 ) / ( H5S_UNLIMITED ) }
      DATA {
      (0): {
            [ 352430272, 25602, 2314, 1442149219, 43629 ],
            1,
            1.44215e+09,
            1442149219,
            0,
            1,
            1,
            0,
            1,
            0,
            0,
            0,
            3,
            0,
            0,
            0,
            0,
            1,
            31095000,
            20000,
            0,
            1000,
            1000, // (this is num_samps)
            1,
            1000,
            [ {
                  1.09068e+09,
                  4.53469e+08
               }, {
                  4.03303e+08,
                  4.11846e+08
               }, {
                  -1.35295e+08,
                  -1.70107e+08
               }, ...

--------------------CODE------------------


typedef struct {
    float f_i;
    float f_q;
} complex_t;

int main(int argc, char *argv[])
{
    std::string path = "/data/hdfFile.be01";
    // Open the file, dataset, and dataspace
    H5::H5File file(path, H5F_ACC_RDONLY);
    H5::DataSet dataset = file.openDataSet("DRx1data");
    H5::DataSpace dataspace = dataset.getSpace();
    
    // Get the number of elements and number of samples
    int nelems = dataspace.getSimpleExtentNpoints();
    int numSamps[nelems];
    
    // Create number of samples type
    H5::CompType sampsType(sizeof(uint));
    sampsType.insertMember("num_samps", 0, H5::PredType::STD_U32LE);
    
    // Read the number of samples
    dataset.read(numSamps, sampsType);
    
    // Now try to read the I/Q data
    H5::CompType iqDataType(sizeof(complex_t));
    iqDataType.insertMember("f_i", HOFFSET(complex_t, f_i), H5::PredType::NATIVE_FLOAT);
    iqDataType.insertMember("f_q", HOFFSET(complex_t, f_q), H5::PredType::NATIVE_FLOAT);
    complex_t** iqData = new complex_t*[nelems];
    for(int i = 0 ; i < nelems ; ++i)
    {
        iqData[i] = new complex_t[numSamps[i]];
    }
    dataset.read(iqData, iqDataType);


_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5


_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Reading Nested Compound objects in C++

Steven Walton
So I don't have too much experience with compound data but I thought I'd take a crack at it. I had an issue a while ago trying to use pointer arrays, hdf5 allowing me to use float* but not float**. There are two ways you can rewrite this, assuming the data isn't rank 1. You could use a ctype array and use malloc to allocate the space, or you can use boost::multi_array. I found the latter to be nicer to work with. Data needs to be contiguous for hdf5 to understand it. If that doesn't work are you able to pull out the whole compound data at once? It appears that you have a nested structure.

If I am reading this right you have
typedef struct
{
  float f_i;
  float f_q;
} iq_data;

typedef struct
{
  bunch of stuff
  iq_data nested;
} DRdata;

You could also always try opening the data with python's h5py. I find the tool useful as it just reads whatever data you throw at it.

On Tue, Jul 12, 2016 at 7:57 AM, Todd Dobmeyer <[hidden email]> wrote:
Binh-Minh

Making your change keeps me from crashing, but unfortunately doesn't get any of the data.  So to better understand my data, I have 7500 pieces of DATA in the DATASPACE. Inside 1 piece of data is an array of Compound objects each containing 2 floats. The Compound object is called iq_data and the 2 floats inside are f_i and f_q.

I originally did the 2D array thinking I needed the outside dimension to be 7500 and the inner dimensions to be 1000. But now I think I need a 1D array that holds 7500*1000 compound objects or 7500*1000*2 floats. However, I cannot figure out how to read the data. Do I read "iq_data" or do I try and read "f_i" and "f_q"?

Thanks again and I hope I clarified things!
Todd

On Tue, Jul 12, 2016 at 2:14 AM, Binh-Minh Ribler <[hidden email]> wrote:

Hi Todd,


I'm not sure I understood your data completely, but I wonder what you will get if you change this line:


complex_t** iqData = new complex_t*[nelems];

to:


complex_t* iqData = new complex_t[nelems];

and read without the for loop...


Binh-Minh



From: Hdf-forum <[hidden email]> on behalf of Todd Dobmeyer <[hidden email]>
Sent: Monday, July 11, 2016 3:10 PM
To: [hidden email]
Subject: [Hdf-forum] Reading Nested Compound objects in C++
 
All

I have an HDF5 file I need to read in a C++ application. I performed an h5dump on the data and have the following structure (with many items removed that I am not interested in). I have successfully read in all 7500 values of "num_samps" using the C++ HDF5 library. What I am struggling to figure out how to read is the array of compound "iq_data". I know in the datatype it says there are 1000 compound objects. I don't know if I can get this value out of the DataType, but I do know the num_samps points to this array size as well, which is fine since I can read this. Can any of you help me figure out how to read the "iq_data" array. Below is the sample data followed by my code I have so far. In my code "numSamps" is good, but "iqData" is not holding anything useful. I am sure I am missing something.

Thanks for your help!
Todd Dobmeyer

--------------SAMPLE DATA-------------------------

HDF5 "hdfFile.be01" {
GROUP "/" {
   DATATYPE "DRdata" H5T_COMPOUND {
      H5T_ARRAY { [5] H5T_STD_U32LE } "guid";
      H5T_STD_I32LE "version";
      ...
      H5T_STD_U32LE "num_samps";
      ...
      H5T_STD_U32LE "sequence_dummy";
      H5T_ARRAY { [1000] H5T_COMPOUND {
         H5T_IEEE_F32LE "f_i";
         H5T_IEEE_F32LE "f_q";
      } } "iq_data";
   }
   DATASET "DRx1data" {
      DATATYPE  "/DRx1"
      DATASPACE  SIMPLE { ( 7500 ) / ( H5S_UNLIMITED ) }
      DATA {
      (0): {
            [ 352430272, 25602, 2314, 1442149219, 43629 ],
            1,
            1.44215e+09,
            1442149219,
            0,
            1,
            1,
            0,
            1,
            0,
            0,
            0,
            3,
            0,
            0,
            0,
            0,
            1,
            31095000,
            20000,
            0,
            1000,
            1000, // (this is num_samps)
            1,
            1000,
            [ {
                  1.09068e+09,
                  4.53469e+08
               }, {
                  4.03303e+08,
                  4.11846e+08
               }, {
                  -1.35295e+08,
                  -1.70107e+08
               }, ...

--------------------CODE------------------


typedef struct {
    float f_i;
    float f_q;
} complex_t;

int main(int argc, char *argv[])
{
    std::string path = "/data/hdfFile.be01";
    // Open the file, dataset, and dataspace
    H5::H5File file(path, H5F_ACC_RDONLY);
    H5::DataSet dataset = file.openDataSet("DRx1data");
    H5::DataSpace dataspace = dataset.getSpace();
    
    // Get the number of elements and number of samples
    int nelems = dataspace.getSimpleExtentNpoints();
    int numSamps[nelems];
    
    // Create number of samples type
    H5::CompType sampsType(sizeof(uint));
    sampsType.insertMember("num_samps", 0, H5::PredType::STD_U32LE);
    
    // Read the number of samples
    dataset.read(numSamps, sampsType);
    
    // Now try to read the I/Q data
    H5::CompType iqDataType(sizeof(complex_t));
    iqDataType.insertMember("f_i", HOFFSET(complex_t, f_i), H5::PredType::NATIVE_FLOAT);
    iqDataType.insertMember("f_q", HOFFSET(complex_t, f_q), H5::PredType::NATIVE_FLOAT);
    complex_t** iqData = new complex_t*[nelems];
    for(int i = 0 ; i < nelems ; ++i)
    {
        iqData[i] = new complex_t[numSamps[i]];
    }
    dataset.read(iqData, iqDataType);


_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5


_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5


_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Reading Nested Compound objects in C++

David-2
Use the DRdata datatype that's already in the file.

On Tue, Jul 12, 2016 at 8:26 AM, Steven Walton <[hidden email]> wrote:
So I don't have too much experience with compound data but I thought I'd take a crack at it. I had an issue a while ago trying to use pointer arrays, hdf5 allowing me to use float* but not float**. There are two ways you can rewrite this, assuming the data isn't rank 1. You could use a ctype array and use malloc to allocate the space, or you can use boost::multi_array. I found the latter to be nicer to work with. Data needs to be contiguous for hdf5 to understand it. If that doesn't work are you able to pull out the whole compound data at once? It appears that you have a nested structure.

If I am reading this right you have
typedef struct
{
  float f_i;
  float f_q;
} iq_data;

typedef struct
{
  bunch of stuff
  iq_data nested;
} DRdata;

You could also always try opening the data with python's h5py. I find the tool useful as it just reads whatever data you throw at it.

On Tue, Jul 12, 2016 at 7:57 AM, Todd Dobmeyer <[hidden email]> wrote:
Binh-Minh

Making your change keeps me from crashing, but unfortunately doesn't get any of the data.  So to better understand my data, I have 7500 pieces of DATA in the DATASPACE. Inside 1 piece of data is an array of Compound objects each containing 2 floats. The Compound object is called iq_data and the 2 floats inside are f_i and f_q.

I originally did the 2D array thinking I needed the outside dimension to be 7500 and the inner dimensions to be 1000. But now I think I need a 1D array that holds 7500*1000 compound objects or 7500*1000*2 floats. However, I cannot figure out how to read the data. Do I read "iq_data" or do I try and read "f_i" and "f_q"?

Thanks again and I hope I clarified things!
Todd

On Tue, Jul 12, 2016 at 2:14 AM, Binh-Minh Ribler <[hidden email]> wrote:

Hi Todd,


I'm not sure I understood your data completely, but I wonder what you will get if you change this line:


complex_t** iqData = new complex_t*[nelems];

to:


complex_t* iqData = new complex_t[nelems];

and read without the for loop...


Binh-Minh



From: Hdf-forum <[hidden email]> on behalf of Todd Dobmeyer <[hidden email]>
Sent: Monday, July 11, 2016 3:10 PM
To: [hidden email]
Subject: [Hdf-forum] Reading Nested Compound objects in C++
 
All

I have an HDF5 file I need to read in a C++ application. I performed an h5dump on the data and have the following structure (with many items removed that I am not interested in). I have successfully read in all 7500 values of "num_samps" using the C++ HDF5 library. What I am struggling to figure out how to read is the array of compound "iq_data". I know in the datatype it says there are 1000 compound objects. I don't know if I can get this value out of the DataType, but I do know the num_samps points to this array size as well, which is fine since I can read this. Can any of you help me figure out how to read the "iq_data" array. Below is the sample data followed by my code I have so far. In my code "numSamps" is good, but "iqData" is not holding anything useful. I am sure I am missing something.

Thanks for your help!
Todd Dobmeyer

--------------SAMPLE DATA-------------------------

HDF5 "hdfFile.be01" {
GROUP "/" {
   DATATYPE "DRdata" H5T_COMPOUND {
      H5T_ARRAY { [5] H5T_STD_U32LE } "guid";
      H5T_STD_I32LE "version";
      ...
      H5T_STD_U32LE "num_samps";
      ...
      H5T_STD_U32LE "sequence_dummy";
      H5T_ARRAY { [1000] H5T_COMPOUND {
         H5T_IEEE_F32LE "f_i";
         H5T_IEEE_F32LE "f_q";
      } } "iq_data";
   }
   DATASET "DRx1data" {
      DATATYPE  "/DRx1"
      DATASPACE  SIMPLE { ( 7500 ) / ( H5S_UNLIMITED ) }
      DATA {
      (0): {
            [ 352430272, 25602, 2314, 1442149219, 43629 ],
            1,
            1.44215e+09,
            1442149219,
            0,
            1,
            1,
            0,
            1,
            0,
            0,
            0,
            3,
            0,
            0,
            0,
            0,
            1,
            31095000,
            20000,
            0,
            1000,
            1000, // (this is num_samps)
            1,
            1000,
            [ {
                  1.09068e+09,
                  4.53469e+08
               }, {
                  4.03303e+08,
                  4.11846e+08
               }, {
                  -1.35295e+08,
                  -1.70107e+08
               }, ...

--------------------CODE------------------


typedef struct {
    float f_i;
    float f_q;
} complex_t;

int main(int argc, char *argv[])
{
    std::string path = "/data/hdfFile.be01";
    // Open the file, dataset, and dataspace
    H5::H5File file(path, H5F_ACC_RDONLY);
    H5::DataSet dataset = file.openDataSet("DRx1data");
    H5::DataSpace dataspace = dataset.getSpace();
    
    // Get the number of elements and number of samples
    int nelems = dataspace.getSimpleExtentNpoints();
    int numSamps[nelems];
    
    // Create number of samples type
    H5::CompType sampsType(sizeof(uint));
    sampsType.insertMember("num_samps", 0, H5::PredType::STD_U32LE);
    
    // Read the number of samples
    dataset.read(numSamps, sampsType);
    
    // Now try to read the I/Q data
    H5::CompType iqDataType(sizeof(complex_t));
    iqDataType.insertMember("f_i", HOFFSET(complex_t, f_i), H5::PredType::NATIVE_FLOAT);
    iqDataType.insertMember("f_q", HOFFSET(complex_t, f_q), H5::PredType::NATIVE_FLOAT);
    complex_t** iqData = new complex_t*[nelems];
    for(int i = 0 ; i < nelems ; ++i)
    {
        iqData[i] = new complex_t[numSamps[i]];
    }
    dataset.read(iqData, iqDataType);


_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5


_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5


_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5


_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Reading Nested Compound objects in C++

Todd Dobmeyer
What do you mean by the DRdata datatype? I am not sure what you mean or how I would use it. This is my first time working with HDF5, sorry for some newbie questions.

On Tue, Jul 12, 2016 at 1:05 PM, David <[hidden email]> wrote:
Use the DRdata datatype that's already in the file.

On Tue, Jul 12, 2016 at 8:26 AM, Steven Walton <[hidden email]> wrote:
So I don't have too much experience with compound data but I thought I'd take a crack at it. I had an issue a while ago trying to use pointer arrays, hdf5 allowing me to use float* but not float**. There are two ways you can rewrite this, assuming the data isn't rank 1. You could use a ctype array and use malloc to allocate the space, or you can use boost::multi_array. I found the latter to be nicer to work with. Data needs to be contiguous for hdf5 to understand it. If that doesn't work are you able to pull out the whole compound data at once? It appears that you have a nested structure.

If I am reading this right you have
typedef struct
{
  float f_i;
  float f_q;
} iq_data;

typedef struct
{
  bunch of stuff
  iq_data nested;
} DRdata;

You could also always try opening the data with python's h5py. I find the tool useful as it just reads whatever data you throw at it.

On Tue, Jul 12, 2016 at 7:57 AM, Todd Dobmeyer <[hidden email]> wrote:
Binh-Minh

Making your change keeps me from crashing, but unfortunately doesn't get any of the data.  So to better understand my data, I have 7500 pieces of DATA in the DATASPACE. Inside 1 piece of data is an array of Compound objects each containing 2 floats. The Compound object is called iq_data and the 2 floats inside are f_i and f_q.

I originally did the 2D array thinking I needed the outside dimension to be 7500 and the inner dimensions to be 1000. But now I think I need a 1D array that holds 7500*1000 compound objects or 7500*1000*2 floats. However, I cannot figure out how to read the data. Do I read "iq_data" or do I try and read "f_i" and "f_q"?

Thanks again and I hope I clarified things!
Todd

On Tue, Jul 12, 2016 at 2:14 AM, Binh-Minh Ribler <[hidden email]> wrote:

Hi Todd,


I'm not sure I understood your data completely, but I wonder what you will get if you change this line:


complex_t** iqData = new complex_t*[nelems];

to:


complex_t* iqData = new complex_t[nelems];

and read without the for loop...


Binh-Minh



From: Hdf-forum <[hidden email]> on behalf of Todd Dobmeyer <[hidden email]>
Sent: Monday, July 11, 2016 3:10 PM
To: [hidden email]
Subject: [Hdf-forum] Reading Nested Compound objects in C++
 
All

I have an HDF5 file I need to read in a C++ application. I performed an h5dump on the data and have the following structure (with many items removed that I am not interested in). I have successfully read in all 7500 values of "num_samps" using the C++ HDF5 library. What I am struggling to figure out how to read is the array of compound "iq_data". I know in the datatype it says there are 1000 compound objects. I don't know if I can get this value out of the DataType, but I do know the num_samps points to this array size as well, which is fine since I can read this. Can any of you help me figure out how to read the "iq_data" array. Below is the sample data followed by my code I have so far. In my code "numSamps" is good, but "iqData" is not holding anything useful. I am sure I am missing something.

Thanks for your help!
Todd Dobmeyer

--------------SAMPLE DATA-------------------------

HDF5 "hdfFile.be01" {
GROUP "/" {
   DATATYPE "DRdata" H5T_COMPOUND {
      H5T_ARRAY { [5] H5T_STD_U32LE } "guid";
      H5T_STD_I32LE "version";
      ...
      H5T_STD_U32LE "num_samps";
      ...
      H5T_STD_U32LE "sequence_dummy";
      H5T_ARRAY { [1000] H5T_COMPOUND {
         H5T_IEEE_F32LE "f_i";
         H5T_IEEE_F32LE "f_q";
      } } "iq_data";
   }
   DATASET "DRx1data" {
      DATATYPE  "/DRx1"
      DATASPACE  SIMPLE { ( 7500 ) / ( H5S_UNLIMITED ) }
      DATA {
      (0): {
            [ 352430272, 25602, 2314, 1442149219, 43629 ],
            1,
            1.44215e+09,
            1442149219,
            0,
            1,
            1,
            0,
            1,
            0,
            0,
            0,
            3,
            0,
            0,
            0,
            0,
            1,
            31095000,
            20000,
            0,
            1000,
            1000, // (this is num_samps)
            1,
            1000,
            [ {
                  1.09068e+09,
                  4.53469e+08
               }, {
                  4.03303e+08,
                  4.11846e+08
               }, {
                  -1.35295e+08,
                  -1.70107e+08
               }, ...

--------------------CODE------------------


typedef struct {
    float f_i;
    float f_q;
} complex_t;

int main(int argc, char *argv[])
{
    std::string path = "/data/hdfFile.be01";
    // Open the file, dataset, and dataspace
    H5::H5File file(path, H5F_ACC_RDONLY);
    H5::DataSet dataset = file.openDataSet("DRx1data");
    H5::DataSpace dataspace = dataset.getSpace();
    
    // Get the number of elements and number of samples
    int nelems = dataspace.getSimpleExtentNpoints();
    int numSamps[nelems];
    
    // Create number of samples type
    H5::CompType sampsType(sizeof(uint));
    sampsType.insertMember("num_samps", 0, H5::PredType::STD_U32LE);
    
    // Read the number of samples
    dataset.read(numSamps, sampsType);
    
    // Now try to read the I/Q data
    H5::CompType iqDataType(sizeof(complex_t));
    iqDataType.insertMember("f_i", HOFFSET(complex_t, f_i), H5::PredType::NATIVE_FLOAT);
    iqDataType.insertMember("f_q", HOFFSET(complex_t, f_q), H5::PredType::NATIVE_FLOAT);
    complex_t** iqData = new complex_t*[nelems];
    for(int i = 0 ; i < nelems ; ++i)
    {
        iqData[i] = new complex_t[numSamps[i]];
    }
    dataset.read(iqData, iqDataType);


_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5


_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5


_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5


_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5


_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Reading Nested Compound objects in C++

Todd Dobmeyer
Okay, sorry I now realized what you meant by the DRdata datatype. I am using that to know I need to get iq_data out but am not certain how to read this data from the dataset. Or are you suggesting there are class functions I can use from the DataType object to easily pull out my data?

On Tue, Jul 12, 2016 at 1:23 PM, Todd Dobmeyer <[hidden email]> wrote:
What do you mean by the DRdata datatype? I am not sure what you mean or how I would use it. This is my first time working with HDF5, sorry for some newbie questions.

On Tue, Jul 12, 2016 at 1:05 PM, David <[hidden email]> wrote:
Use the DRdata datatype that's already in the file.

On Tue, Jul 12, 2016 at 8:26 AM, Steven Walton <[hidden email]> wrote:
So I don't have too much experience with compound data but I thought I'd take a crack at it. I had an issue a while ago trying to use pointer arrays, hdf5 allowing me to use float* but not float**. There are two ways you can rewrite this, assuming the data isn't rank 1. You could use a ctype array and use malloc to allocate the space, or you can use boost::multi_array. I found the latter to be nicer to work with. Data needs to be contiguous for hdf5 to understand it. If that doesn't work are you able to pull out the whole compound data at once? It appears that you have a nested structure.

If I am reading this right you have
typedef struct
{
  float f_i;
  float f_q;
} iq_data;

typedef struct
{
  bunch of stuff
  iq_data nested;
} DRdata;

You could also always try opening the data with python's h5py. I find the tool useful as it just reads whatever data you throw at it.

On Tue, Jul 12, 2016 at 7:57 AM, Todd Dobmeyer <[hidden email]> wrote:
Binh-Minh

Making your change keeps me from crashing, but unfortunately doesn't get any of the data.  So to better understand my data, I have 7500 pieces of DATA in the DATASPACE. Inside 1 piece of data is an array of Compound objects each containing 2 floats. The Compound object is called iq_data and the 2 floats inside are f_i and f_q.

I originally did the 2D array thinking I needed the outside dimension to be 7500 and the inner dimensions to be 1000. But now I think I need a 1D array that holds 7500*1000 compound objects or 7500*1000*2 floats. However, I cannot figure out how to read the data. Do I read "iq_data" or do I try and read "f_i" and "f_q"?

Thanks again and I hope I clarified things!
Todd

On Tue, Jul 12, 2016 at 2:14 AM, Binh-Minh Ribler <[hidden email]> wrote:

Hi Todd,


I'm not sure I understood your data completely, but I wonder what you will get if you change this line:


complex_t** iqData = new complex_t*[nelems];

to:


complex_t* iqData = new complex_t[nelems];

and read without the for loop...


Binh-Minh



From: Hdf-forum <[hidden email]> on behalf of Todd Dobmeyer <[hidden email]>
Sent: Monday, July 11, 2016 3:10 PM
To: [hidden email]
Subject: [Hdf-forum] Reading Nested Compound objects in C++
 
All

I have an HDF5 file I need to read in a C++ application. I performed an h5dump on the data and have the following structure (with many items removed that I am not interested in). I have successfully read in all 7500 values of "num_samps" using the C++ HDF5 library. What I am struggling to figure out how to read is the array of compound "iq_data". I know in the datatype it says there are 1000 compound objects. I don't know if I can get this value out of the DataType, but I do know the num_samps points to this array size as well, which is fine since I can read this. Can any of you help me figure out how to read the "iq_data" array. Below is the sample data followed by my code I have so far. In my code "numSamps" is good, but "iqData" is not holding anything useful. I am sure I am missing something.

Thanks for your help!
Todd Dobmeyer

--------------SAMPLE DATA-------------------------

HDF5 "hdfFile.be01" {
GROUP "/" {
   DATATYPE "DRdata" H5T_COMPOUND {
      H5T_ARRAY { [5] H5T_STD_U32LE } "guid";
      H5T_STD_I32LE "version";
      ...
      H5T_STD_U32LE "num_samps";
      ...
      H5T_STD_U32LE "sequence_dummy";
      H5T_ARRAY { [1000] H5T_COMPOUND {
         H5T_IEEE_F32LE "f_i";
         H5T_IEEE_F32LE "f_q";
      } } "iq_data";
   }
   DATASET "DRx1data" {
      DATATYPE  "/DRx1"
      DATASPACE  SIMPLE { ( 7500 ) / ( H5S_UNLIMITED ) }
      DATA {
      (0): {
            [ 352430272, 25602, 2314, 1442149219, 43629 ],
            1,
            1.44215e+09,
            1442149219,
            0,
            1,
            1,
            0,
            1,
            0,
            0,
            0,
            3,
            0,
            0,
            0,
            0,
            1,
            31095000,
            20000,
            0,
            1000,
            1000, // (this is num_samps)
            1,
            1000,
            [ {
                  1.09068e+09,
                  4.53469e+08
               }, {
                  4.03303e+08,
                  4.11846e+08
               }, {
                  -1.35295e+08,
                  -1.70107e+08
               }, ...

--------------------CODE------------------


typedef struct {
    float f_i;
    float f_q;
} complex_t;

int main(int argc, char *argv[])
{
    std::string path = "/data/hdfFile.be01";
    // Open the file, dataset, and dataspace
    H5::H5File file(path, H5F_ACC_RDONLY);
    H5::DataSet dataset = file.openDataSet("DRx1data");
    H5::DataSpace dataspace = dataset.getSpace();
    
    // Get the number of elements and number of samples
    int nelems = dataspace.getSimpleExtentNpoints();
    int numSamps[nelems];
    
    // Create number of samples type
    H5::CompType sampsType(sizeof(uint));
    sampsType.insertMember("num_samps", 0, H5::PredType::STD_U32LE);
    
    // Read the number of samples
    dataset.read(numSamps, sampsType);
    
    // Now try to read the I/Q data
    H5::CompType iqDataType(sizeof(complex_t));
    iqDataType.insertMember("f_i", HOFFSET(complex_t, f_i), H5::PredType::NATIVE_FLOAT);
    iqDataType.insertMember("f_q", HOFFSET(complex_t, f_q), H5::PredType::NATIVE_FLOAT);
    complex_t** iqData = new complex_t*[nelems];
    for(int i = 0 ; i < nelems ; ++i)
    {
        iqData[i] = new complex_t[numSamps[i]];
    }
    dataset.read(iqData, iqDataType);


_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5


_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5


_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5


_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5



_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Reading Nested Compound objects in C++

David-2
> Or are you suggesting there are class functions I can use from the DataType object to easily pull out my data?

Yes. HDF5 has an API for creating custom data types: HDF5 Datatypes
The 'DATATYPE "DRdata"' section in your file is the custom datatype used to write the data. You can use it to read the data.


On Tue, Jul 12, 2016 at 10:30 AM, Todd Dobmeyer <[hidden email]> wrote:
Okay, sorry I now realized what you meant by the DRdata datatype. I am using that to know I need to get iq_data out but am not certain how to read this data from the dataset. Or are you suggesting there are class functions I can use from the DataType object to easily pull out my data?

On Tue, Jul 12, 2016 at 1:23 PM, Todd Dobmeyer <[hidden email]> wrote:
What do you mean by the DRdata datatype? I am not sure what you mean or how I would use it. This is my first time working with HDF5, sorry for some newbie questions.

On Tue, Jul 12, 2016 at 1:05 PM, David <[hidden email]> wrote:
Use the DRdata datatype that's already in the file.

On Tue, Jul 12, 2016 at 8:26 AM, Steven Walton <[hidden email]> wrote:
So I don't have too much experience with compound data but I thought I'd take a crack at it. I had an issue a while ago trying to use pointer arrays, hdf5 allowing me to use float* but not float**. There are two ways you can rewrite this, assuming the data isn't rank 1. You could use a ctype array and use malloc to allocate the space, or you can use boost::multi_array. I found the latter to be nicer to work with. Data needs to be contiguous for hdf5 to understand it. If that doesn't work are you able to pull out the whole compound data at once? It appears that you have a nested structure.

If I am reading this right you have
typedef struct
{
  float f_i;
  float f_q;
} iq_data;

typedef struct
{
  bunch of stuff
  iq_data nested;
} DRdata;

You could also always try opening the data with python's h5py. I find the tool useful as it just reads whatever data you throw at it.

On Tue, Jul 12, 2016 at 7:57 AM, Todd Dobmeyer <[hidden email]> wrote:
Binh-Minh

Making your change keeps me from crashing, but unfortunately doesn't get any of the data.  So to better understand my data, I have 7500 pieces of DATA in the DATASPACE. Inside 1 piece of data is an array of Compound objects each containing 2 floats. The Compound object is called iq_data and the 2 floats inside are f_i and f_q.

I originally did the 2D array thinking I needed the outside dimension to be 7500 and the inner dimensions to be 1000. But now I think I need a 1D array that holds 7500*1000 compound objects or 7500*1000*2 floats. However, I cannot figure out how to read the data. Do I read "iq_data" or do I try and read "f_i" and "f_q"?

Thanks again and I hope I clarified things!
Todd

On Tue, Jul 12, 2016 at 2:14 AM, Binh-Minh Ribler <[hidden email]> wrote:

Hi Todd,


I'm not sure I understood your data completely, but I wonder what you will get if you change this line:


complex_t** iqData = new complex_t*[nelems];

to:


complex_t* iqData = new complex_t[nelems];

and read without the for loop...


Binh-Minh



From: Hdf-forum <[hidden email]> on behalf of Todd Dobmeyer <[hidden email]>
Sent: Monday, July 11, 2016 3:10 PM
To: [hidden email]
Subject: [Hdf-forum] Reading Nested Compound objects in C++
 
All

I have an HDF5 file I need to read in a C++ application. I performed an h5dump on the data and have the following structure (with many items removed that I am not interested in). I have successfully read in all 7500 values of "num_samps" using the C++ HDF5 library. What I am struggling to figure out how to read is the array of compound "iq_data". I know in the datatype it says there are 1000 compound objects. I don't know if I can get this value out of the DataType, but I do know the num_samps points to this array size as well, which is fine since I can read this. Can any of you help me figure out how to read the "iq_data" array. Below is the sample data followed by my code I have so far. In my code "numSamps" is good, but "iqData" is not holding anything useful. I am sure I am missing something.

Thanks for your help!
Todd Dobmeyer

--------------SAMPLE DATA-------------------------

HDF5 "hdfFile.be01" {
GROUP "/" {
   DATATYPE "DRdata" H5T_COMPOUND {
      H5T_ARRAY { [5] H5T_STD_U32LE } "guid";
      H5T_STD_I32LE "version";
      ...
      H5T_STD_U32LE "num_samps";
      ...
      H5T_STD_U32LE "sequence_dummy";
      H5T_ARRAY { [1000] H5T_COMPOUND {
         H5T_IEEE_F32LE "f_i";
         H5T_IEEE_F32LE "f_q";
      } } "iq_data";
   }
   DATASET "DRx1data" {
      DATATYPE  "/DRx1"
      DATASPACE  SIMPLE { ( 7500 ) / ( H5S_UNLIMITED ) }
      DATA {
      (0): {
            [ 352430272, 25602, 2314, 1442149219, 43629 ],
            1,
            1.44215e+09,
            1442149219,
            0,
            1,
            1,
            0,
            1,
            0,
            0,
            0,
            3,
            0,
            0,
            0,
            0,
            1,
            31095000,
            20000,
            0,
            1000,
            1000, // (this is num_samps)
            1,
            1000,
            [ {
                  1.09068e+09,
                  4.53469e+08
               }, {
                  4.03303e+08,
                  4.11846e+08
               }, {
                  -1.35295e+08,
                  -1.70107e+08
               }, ...

--------------------CODE------------------


typedef struct {
    float f_i;
    float f_q;
} complex_t;

int main(int argc, char *argv[])
{
    std::string path = "/data/hdfFile.be01";
    // Open the file, dataset, and dataspace
    H5::H5File file(path, H5F_ACC_RDONLY);
    H5::DataSet dataset = file.openDataSet("DRx1data");
    H5::DataSpace dataspace = dataset.getSpace();
    
    // Get the number of elements and number of samples
    int nelems = dataspace.getSimpleExtentNpoints();
    int numSamps[nelems];
    
    // Create number of samples type
    H5::CompType sampsType(sizeof(uint));
    sampsType.insertMember("num_samps", 0, H5::PredType::STD_U32LE);
    
    // Read the number of samples
    dataset.read(numSamps, sampsType);
    
    // Now try to read the I/Q data
    H5::CompType iqDataType(sizeof(complex_t));
    iqDataType.insertMember("f_i", HOFFSET(complex_t, f_i), H5::PredType::NATIVE_FLOAT);
    iqDataType.insertMember("f_q", HOFFSET(complex_t, f_q), H5::PredType::NATIVE_FLOAT);
    complex_t** iqData = new complex_t*[nelems];
    for(int i = 0 ; i < nelems ; ++i)
    {
        iqData[i] = new complex_t[numSamps[i]];
    }
    dataset.read(iqData, iqDataType);


_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5


_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5


_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5


_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5



_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5


_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Reading Nested Compound objects in C++

Binh-Minh Ribler
In reply to this post by Todd Dobmeyer

Hi Todd,


I think you should try to read iq_data.  Your buffer is probably ok to be a 2-dim array of the compound type.

For the datatype, can you do ArrayType of ArrayType of CompType?  Roughly:


CompType comp_t...

ArrayType arr_in_t(cmp_t,...)

ArrayType arr_out_t(arr_in_t,...)

Then, read(yourbuffer, comp_t)


I haven't tried it so I'm not sure if it's allowed, but I would try that first.


Binh-Minh



From: Hdf-forum <[hidden email]> on behalf of Todd Dobmeyer <[hidden email]>
Sent: Tuesday, July 12, 2016 8:57 AM
To: HDF Users Discussion List
Subject: Re: [Hdf-forum] Reading Nested Compound objects in C++
 
Binh-Minh

Making your change keeps me from crashing, but unfortunately doesn't get any of the data.  So to better understand my data, I have 7500 pieces of DATA in the DATASPACE. Inside 1 piece of data is an array of Compound objects each containing 2 floats. The Compound object is called iq_data and the 2 floats inside are f_i and f_q.

I originally did the 2D array thinking I needed the outside dimension to be 7500 and the inner dimensions to be 1000. But now I think I need a 1D array that holds 7500*1000 compound objects or 7500*1000*2 floats. However, I cannot figure out how to read the data. Do I read "iq_data" or do I try and read "f_i" and "f_q"?

Thanks again and I hope I clarified things!
Todd

On Tue, Jul 12, 2016 at 2:14 AM, Binh-Minh Ribler <[hidden email]> wrote:

Hi Todd,


I'm not sure I understood your data completely, but I wonder what you will get if you change this line:


complex_t** iqData = new complex_t*[nelems];

to:


complex_t* iqData = new complex_t[nelems];

and read without the for loop...


Binh-Minh



From: Hdf-forum <[hidden email]> on behalf of Todd Dobmeyer <[hidden email]>
Sent: Monday, July 11, 2016 3:10 PM
To: [hidden email]
Subject: [Hdf-forum] Reading Nested Compound objects in C++
 
All

I have an HDF5 file I need to read in a C++ application. I performed an h5dump on the data and have the following structure (with many items removed that I am not interested in). I have successfully read in all 7500 values of "num_samps" using the C++ HDF5 library. What I am struggling to figure out how to read is the array of compound "iq_data". I know in the datatype it says there are 1000 compound objects. I don't know if I can get this value out of the DataType, but I do know the num_samps points to this array size as well, which is fine since I can read this. Can any of you help me figure out how to read the "iq_data" array. Below is the sample data followed by my code I have so far. In my code "numSamps" is good, but "iqData" is not holding anything useful. I am sure I am missing something.

Thanks for your help!
Todd Dobmeyer

--------------SAMPLE DATA-------------------------

HDF5 "hdfFile.be01" {
GROUP "/" {
   DATATYPE "DRdata" H5T_COMPOUND {
      H5T_ARRAY { [5] H5T_STD_U32LE } "guid";
      H5T_STD_I32LE "version";
      ...
      H5T_STD_U32LE "num_samps";
      ...
      H5T_STD_U32LE "sequence_dummy";
      H5T_ARRAY { [1000] H5T_COMPOUND {
         H5T_IEEE_F32LE "f_i";
         H5T_IEEE_F32LE "f_q";
      } } "iq_data";
   }
   DATASET "DRx1data" {
      DATATYPE  "/DRx1"
      DATASPACE  SIMPLE { ( 7500 ) / ( H5S_UNLIMITED ) }
      DATA {
      (0): {
            [ 352430272, 25602, 2314, 1442149219, 43629 ],
            1,
            1.44215e+09,
            1442149219,
            0,
            1,
            1,
            0,
            1,
            0,
            0,
            0,
            3,
            0,
            0,
            0,
            0,
            1,
            31095000,
            20000,
            0,
            1000,
            1000, // (this is num_samps)
            1,
            1000,
            [ {
                  1.09068e+09,
                  4.53469e+08
               }, {
                  4.03303e+08,
                  4.11846e+08
               }, {
                  -1.35295e+08,
                  -1.70107e+08
               }, ...

--------------------CODE------------------


typedef struct {
    float f_i;
    float f_q;
} complex_t;

int main(int argc, char *argv[])
{
    std::string path = "/data/hdfFile.be01";
    // Open the file, dataset, and dataspace
    H5::H5File file(path, H5F_ACC_RDONLY);
    H5::DataSet dataset = file.openDataSet("DRx1data");
    H5::DataSpace dataspace = dataset.getSpace();
    
    // Get the number of elements and number of samples
    int nelems = dataspace.getSimpleExtentNpoints();
    int numSamps[nelems];
    
    // Create number of samples type
    H5::CompType sampsType(sizeof(uint));
    sampsType.insertMember("num_samps", 0, H5::PredType::STD_U32LE);
    
    // Read the number of samples
    dataset.read(numSamps, sampsType);
    
    // Now try to read the I/Q data
    H5::CompType iqDataType(sizeof(complex_t));
    iqDataType.insertMember("f_i", HOFFSET(complex_t, f_i), H5::PredType::NATIVE_FLOAT);
    iqDataType.insertMember("f_q", HOFFSET(complex_t, f_q), H5::PredType::NATIVE_FLOAT);
    complex_t** iqData = new complex_t*[nelems];
    for(int i = 0 ; i < nelems ; ++i)
    {
        iqData[i] = new complex_t[numSamps[i]];
    }
    dataset.read(iqData, iqDataType);


_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5


_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
Loading...