H5T_ARRAY

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

H5T_ARRAY

Vladyslav Danilov

Hello,

my name is Vlad and HDF5 is new for me.
I want to ask an advise in one case.
i have a datatype and there i have smth like:

DATATYPE  H5T_COMPOUND {
         H5T_STD_U32LE "fillnum";
         H5T_STD_U32LE "runnum";
         H5T_STD_U32LE "lsnum";
         H5T_STD_U32LE "nbnum";
         H5T_STD_U32LE "timestampsec";
         H5T_STD_U32LE "timestampmsec";
         H5T_STD_U8LE "publishnnb";
         H5T_STD_U8LE "datasourceid";
         H5T_STD_U8LE "algoid";
         H5T_STD_U8LE "channelid";
         H5T_ARRAY { [3564] H5T_STD_U16LE } "data";
      }

to read it i'm creating a structure and then read the file into it, i define the structure in my code like this:
typedef struct {
      unsigned int      fillnum;
      unsigned int      runnum;
      unsigned int      lsnum;
      unsigned int      nbnum;
      unsigned int      timestampsec;
      unsigned int      timestampmsec;
      unsigned char     publishnnb;
      unsigned char     datasourceid;
      unsigned char     algoid;
      unsigned char     channelid;
      unsigned short     data[3564];
    } bcm1fagghist_type;
but looks like for H5T_ARRAY line 'unsigned short data[3564];' is wrong.
could You advise me how to read it?


Kind regards,
  Vlad Danilov

_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
Reply | Threaded
Open this post in threaded view
|

Re: ***SPAM*** H5T_ARRAY

Rafal Lichwala
Hello Vlad,

Please send me example HDF5 file and your current C code which you are
trying to use to read that file and I will try to find a solution.

Regards,
Rafal


W dniu 2017-11-19 o 19:46, Vlad Danilov pisze:

>
> Hello,
>
> my name is Vlad and HDF5 is new for me.
> I want to ask an advise in one case.
> i have a datatype and there i have smth like:
>
> DATATYPE  H5T_COMPOUND {
>           H5T_STD_U32LE "fillnum";
>           H5T_STD_U32LE "runnum";
>           H5T_STD_U32LE "lsnum";
>           H5T_STD_U32LE "nbnum";
>           H5T_STD_U32LE "timestampsec";
>           H5T_STD_U32LE "timestampmsec";
>           H5T_STD_U8LE "publishnnb";
>           H5T_STD_U8LE "datasourceid";
>           H5T_STD_U8LE "algoid";
>           H5T_STD_U8LE "channelid";
>           H5T_ARRAY { [3564] H5T_STD_U16LE } "data";
>        }
>
> to read it i'm creating a structure and then read the file into it, i
> define the structure in my code like this:
> typedef struct {
>        unsigned int      fillnum;
>        unsigned int      runnum;
>        unsigned int      lsnum;
>        unsigned int      nbnum;
>        unsigned int      timestampsec;
>        unsigned int      timestampmsec;
>        unsigned char     publishnnb;
>        unsigned char     datasourceid;
>        unsigned char     algoid;
>        unsigned char     channelid;
>        unsigned short     data[3564];
>      } bcm1fagghist_type;
> but looks like for H5T_ARRAY line 'unsigned short data[3564];' is wrong.
> could You advise me how to read it?
> ------------------------------------------------------------------------
>
> Kind regards,
>    Vlad Danilov
>
>
> _______________________________________________
> Hdf-forum is for HDF software users discussion.
> [hidden email]
> http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
> Twitter: https://twitter.com/hdf5
>


_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
Reply | Threaded
Open this post in threaded view
|

Re: ***SPAM*** H5T_ARRAY

Vladyslav Danilov
Hello Rafal,
thank You for Your reply.

I found that i can use 'H5Tget_native_type' function, but i have another error.

First - the code itself:

#include "hdf5.h"
#include <stdio.h>
#include <stdlib.h>

#define H5FILE_NAME  "6336_305739_1710261824_1710261836.h5" //"SDSextendible.h5" //      
#define DATASETNAME "ExtendibleArray"
#define RANK         2
#define RANKC        1
#define NX           10
#define NY           5

#define TOPIC        "topic"
#define TIMESTAMPSEC "timestampsec"
#define AVGRAW       "avgraw"
#define RUNNUM       "runnum"
#define LSNUM        "lsnum"
#define NBNUM        "nbnum"

int
main (void)
{
        
    typedef struct {
      unsigned int      fillnum;
      unsigned int      runnum;
      unsigned int      lsnum;
      unsigned int      nbnum;
      unsigned int      timestampsec;
      unsigned int      timestampmsec;
      unsigned char     publishnnb;
      unsigned char     datasourceid;
      unsigned char     algoid;
      unsigned char     channelid;
      unsigned short     data;
    } bcm1fagghist_type;
    
    hid_t       dataSetName;
    hid_t       file;                        /* handles */
    hid_t       dataset;
    hid_t       filespace;
    hid_t       memspace, memtype;
    hid_t       cparms,file_s1_t;
    
    
    hid_t       var_id;
    int         s2_t;
    
    hsize_t     dims[200];                     /* dataset and chunk dimensions*/
    hsize_t     chunk_dims[2];
    hsize_t     col_dims[1];
    hsize_t     count[2];
    hsize_t     offset[2];
    size_t      size;
 /**********************************************/
 bcm1fagghist_type  *read_data;
 /**********************************************/
    herr_t      status, status_n, status_k;

    int         data_out[NX][NY];  /* buffer for dataset to be read */
    int         chunk_out[2][5];   /* buffer for chunk to be read */
    int         column[10];        /* buffer for column to be read */
    int         rank, rank_chunk;
    int        i, j;
    int         data[144];
    
    /*
     * Open the file and the dataset.
     */
    
    file = H5Fopen(H5FILE_NAME, H5F_ACC_RDONLY, H5P_DEFAULT);
    dataset = H5Dopen2(file, "/bcm1fagghist", H5P_DEFAULT);

    /*
     * Get dataset rank and dimension.
     */

    filespace = H5Dget_space(dataset);    /* Get filespace handle first. */
    rank      = H5Sget_simple_extent_ndims(filespace);
    
    status_n  = H5Sget_simple_extent_dims(filespace, dims, NULL);
    
    read_data = (bcm1fagghist_type*) malloc(dims[0]*sizeof(bcm1fagghist_type));
    printf("dataset rank %d, dimensions %lu x %lu, filespace %d\n", rank, (unsigned long)(dims[0]), (unsigned long)(dims[1]), filespace);
    
    /*
     * Define the memory space to read dataset.
     */
    
    //memtype  = H5Tcreate(H5T_COMPOUND, sizeof (bcm1fagghist_type));
    file_s1_t  = H5Dget_type(dataset);  
    memtype  = H5Tget_native_type(file_s1_t, H5T_DIR_DEFAULT);
    memspace = H5Screate_simple(rank,dims,0);

    /*
     * Read dataset back and display.
     */
    
    printf("     >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>            var_id = H5Tinsert(memspace, fillnum, 0, H5T_STD_U32LE);  -  IS  DONE\n");
    status = H5Dread(dataset, memtype,H5S_ALL, H5S_ALL,/* memspace, filespace, */H5P_DEFAULT, read_data);
    printf("     >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>   status = H5Dread(dataset, H5T_NATIVE_INT, memspace, filespace, H5P_DEFAULT, data_out);    -  is DONE\n");
    printf("\n");
    printf("Dataset: \n");
    for (j = 0; j < dims[0]; j++) {
    printf("%d ", read_data[j].fillnum);        
    printf("\n");
        if (j == 10) break;
    }

it's not full code but the problem part. I have an errors in the line  status = H5Dread(dataset, memtype,H5S_ALL, H5S_ALL,/* memspace, filespace, */H5P_DEFAULT, read_data);
The errors itself:

 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>            var_id = H5Tinsert(memspace, fillnum, 0, H5T_STD_U32LE);  -  IS  DONE
HDF5-DIAG: Error detected in HDF5 (1.8.19) thread 0:
  #000: H5Dio.c line 223 in H5Dread(): can't read data
    major: Dataset
    minor: Read failed
  #001: H5Dio.c line 605 in H5D__read(): can't read data
    major: Dataset
    minor: Read failed
  #002: H5Dchunk.c line 2092 in H5D__chunk_read(): unable to read raw data chunk
    major: Low-level I/O
    minor: Read failed
  #003: H5Dchunk.c line 3122 in H5D__chunk_lock(): data pipeline read failed
    major: Data filters
    minor: Filter operation failed
  #004: H5Z.c line 1345 in H5Z_pipeline(): required filter 'blosc' is not registered
    major: Data filters
    minor: Read failed
  #005: H5PL.c line 380 in H5PL_load(): search in paths failed
    major: Plugin for dynamically loaded library
    minor: Can't get value
  #006: H5PL.c line 738 in H5PL__find(): can't open directory: /usr/local/hdf5/lib/plugin
    major: Plugin for dynamically loaded library
    minor: Can't open directory or file
     >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>   status = H5Dread(dataset, H5T_NATIVE_INT, memspace, filespace, H5P_DEFAULT, data_out);    -  is DONE


I would be very glad if you could give an advise in that case.
It seams like i have a wrong path to a filter 'blosc', but i can't find where the filters are.
After installation i passed all tests.

Kind regards,
  Vlad



Понедельник, 20 ноября 2017, 15:30 +01:00 от Rafal Lichwala <[hidden email]>:

Hello Vlad,

Please send me example HDF5 file and your current C code which you are
trying to use to read that file and I will try to find a solution.

Regards,
Rafal


W dniu 2017-11-19 o 19:46, Vlad Danilov pisze:
>
> Hello,
>
> my name is Vlad and HDF5 is new for me.
> I want to ask an advise in one case.
> i have a datatype and there i have smth like:
>
> DATATYPE  H5T_COMPOUND {
>          H5T_STD_U32LE "fillnum";
>          H5T_STD_U32LE "runnum";
>          H5T_STD_U32LE "lsnum";
>          H5T_STD_U32LE "nbnum";
>          H5T_STD_U32LE "timestampsec";
>          H5T_STD_U32LE "timestampmsec";
>          H5T_STD_U8LE "publishnnb";
>          H5T_STD_U8LE "datasourceid";
>          H5T_STD_U8LE "algoid";
>          H5T_STD_U8LE "channelid";
>          H5T_ARRAY { [3564] H5T_STD_U16LE } "data";
>       }
>
> to read it i'm creating a structure and then read the file into it, i
> define the structure in my code like this:
> typedef struct {
>       unsigned int      fillnum;
>       unsigned int      runnum;
>       unsigned int      lsnum;
>       unsigned int      nbnum;
>       unsigned int      timestampsec;
>       unsigned int      timestampmsec;
>       unsigned char     publishnnb;
>       unsigned char     datasourceid;
>       unsigned char     algoid;
>       unsigned char     channelid;
>       unsigned short     data[3564];
>     } bcm1fagghist_type;
> but looks like for H5T_ARRAY line 'unsigned short data[3564];' is wrong.
> could You advise me how to read it?
> ------------------------------------------------------------------------
>
> Kind regards,
>   Vlad Danilov
>
>
> _______________________________________________
> Hdf-forum is for HDF software users discussion.
> [hidden email]
> http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
> Twitter: https://twitter.com/hdf5
>


_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5


Kind regards,
  Vlad Danilov

_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
Reply | Threaded
Open this post in threaded view
|

Re: ***SPAM*** H5T_ARRAY

Vladyslav Danilov
In reply to this post by Rafal Lichwala

The problem solved, i just had to install 'blosc' filter and put it into default dir /usr/local/hdf5/lib/plugin

But i have a question, maybe it's a bit offtopic, but still - does somebody knows how to read PyTables in hdf5 format using C API?

thank You


Понедельник, 20 ноября 2017, 15:30 +01:00 от Rafal Lichwala <[hidden email]>:

Hello Vlad,

Please send me example HDF5 file and your current C code which you are
trying to use to read that file and I will try to find a solution.

Regards,
Rafal


W dniu 2017-11-19 o 19:46, Vlad Danilov pisze:
>
> Hello,
>
> my name is Vlad and HDF5 is new for me.
> I want to ask an advise in one case.
> i have a datatype and there i have smth like:
>
> DATATYPE  H5T_COMPOUND {
>          H5T_STD_U32LE "fillnum";
>          H5T_STD_U32LE "runnum";
>          H5T_STD_U32LE "lsnum";
>          H5T_STD_U32LE "nbnum";
>          H5T_STD_U32LE "timestampsec";
>          H5T_STD_U32LE "timestampmsec";
>          H5T_STD_U8LE "publishnnb";
>          H5T_STD_U8LE "datasourceid";
>          H5T_STD_U8LE "algoid";
>          H5T_STD_U8LE "channelid";
>          H5T_ARRAY { [3564] H5T_STD_U16LE } "data";
>       }
>
> to read it i'm creating a structure and then read the file into it, i
> define the structure in my code like this:
> typedef struct {
>       unsigned int      fillnum;
>       unsigned int      runnum;
>       unsigned int      lsnum;
>       unsigned int      nbnum;
>       unsigned int      timestampsec;
>       unsigned int      timestampmsec;
>       unsigned char     publishnnb;
>       unsigned char     datasourceid;
>       unsigned char     algoid;
>       unsigned char     channelid;
>       unsigned short     data[3564];
>     } bcm1fagghist_type;
> but looks like for H5T_ARRAY line 'unsigned short data[3564];' is wrong.
> could You advise me how to read it?
> ------------------------------------------------------------------------
>
> Kind regards,
>   Vlad Danilov
>
>
> _______________________________________________
> Hdf-forum is for HDF software users discussion.
> [hidden email]
> http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
> Twitter: https://twitter.com/hdf5
>


_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5


Kind regards,
  Vlad Danilov

_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
Reply | Threaded
Open this post in threaded view
|

Re: ***SPAM*** H5T_ARRAY

bljones

Hi Vlad,

 

This PyTables page shows the output of a Pytables file (tutorial1.h5) using h5ls:

http://www.pytables.org/usersguide/tutorials.html

 

It is a compound datatype.

 

On the Examples by API page are C examples:

https://support.hdfgroup.org/HDF5/examples/api-c.html

 

Under “Datatypes” is an example to “Read / Write Compound Datatypes (Dataset):  h5ex_t_cmpd.c

 

Also, the High Level Table (H5TB) APIs allow you how to create and modify simple tables (compound datatypes) in HDF5. There is a tutorial here, in case that might help:

 

https://support.hdfgroup.org/HDF5/Tutor/h5table.html

 

-Barbara

 

 

From: Hdf-forum [mailto:[hidden email]] On Behalf Of Vlad Danilov
Sent: Tuesday, November 21, 2017 8:58 AM
To: HDF Users Discussion List
Subject: Re: [Hdf-forum] ***SPAM*** H5T_ARRAY

 

The problem solved, i just had to install 'blosc' filter and put it into default dir /usr/local/hdf5/lib/plugin

But i have a question, maybe it's a bit offtopic, but still - does somebody knows how to read PyTables in hdf5 format using C API?

thank You

Понедельник, 20 ноября 2017, 15:30 +01:00 от Rafal Lichwala <[hidden email]>:

Hello Vlad,

Please send me example HDF5 file and your current C code which you are
trying to use to read that file and I will try to find a solution.

Regards,
Rafal


W dniu 2017-11-19 o 19:46, Vlad Danilov pisze:
>
> Hello,
>
> my name is Vlad and HDF5 is new for me.
> I want to ask an advise in one case.
> i have a datatype and there i have smth like:
>
> DATATYPE  H5T_COMPOUND {
>          H5T_STD_U32LE "fillnum";
>          H5T_STD_U32LE "runnum";
>          H5T_STD_U32LE "lsnum";
>          H5T_STD_U32LE "nbnum";
>          H5T_STD_U32LE "timestampsec";
>          H5T_STD_U32LE "timestampmsec";
>          H5T_STD_U8LE "publishnnb";
>          H5T_STD_U8LE "datasourceid";
>          H5T_STD_U8LE "algoid";
>          H5T_STD_U8LE "channelid";
>          H5T_ARRAY { [3564] H5T_STD_U16LE } "data";
>       }
>
> to read it i'm creating a structure and then read the file into it, i
> define the structure in my code like this:
> typedef struct {
>       unsigned int      fillnum;
>       unsigned int      runnum;
>       unsigned int      lsnum;
>       unsigned int      nbnum;
>       unsigned int      timestampsec;
>       unsigned int      timestampmsec;
>       unsigned char     publishnnb;
>       unsigned char     datasourceid;
>       unsigned char     algoid;
>       unsigned char     channelid;
>       unsigned short     data[3564];
>     } bcm1fagghist_type;
> but looks like for H5T_ARRAY line 'unsigned short data[3564];' is wrong.
> could You advise me how to read it?
> ------------------------------------------------------------------------
>
> Kind regards,
>   Vlad Danilov
>
>
> _______________________________________________
> Hdf-forum is for HDF software users discussion.
> [hidden email]
> http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
> Twitter: https://twitter.com/hdf5
>


_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5



Kind regards,
  Vlad Danilov


_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5