writing std::strings to an hdf5 file

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

writing std::strings to an hdf5 file

Michael Barrucco
Hello Folks:

I am trying to write a string from a process in which the string changes at each iteration, and i dont know the size of the resulting string array at the time the write needs to happen.

I have some code, however, it only write the first character of the string on each iteration. Here is some sample code.


initialization============================================


std::string file_string = “foo.h5”;

H5File _file( file_name, H5F_ACC_TRUNC );

_col_chunk=0;


    DSetCreatPropList cparms_str;

    hsize_t chunk_dims_str[2] = {1,1};

    cparms_str.setChunk(2, chunk_dims_str);

    hid_t datatype = H5Tcopy(H5T_C_S1);

    char fill_val_str[1] = {""};

    cparms_str.setFillValue( datatype, &fill_val_str);

const H5std_string dataset_state(“string_data");

_dataset_state = _file.createDataSet(dataset_state,

                                         datatype, mspace_s, cparms_str);


Writing within the process===================================================


Get current string

hsize_t dimsext_s[2] = {1, 1};

const char * current_string_char = current_string.c_str();

size_s[0] = dimsext_s[0];

    size_s[1] = dimsext_s[1] + _col_chunk;

DataSpace filespace_state(_dataset_state.getSpace());

     filespace_state.selectHyperslab(H5S_SELECT_SET, dimsext_s, offset_s);

     DataSpace memspace_state(2, dimsext_s, NULL);

     hid_t datatype = H5Tcopy(H5T_C_S1);

    _dataset_state.write(currrent_string_char, datatype, memspace_state, filespace_state);


loading the resulting file==============================

In [1]: import numpy as np

In [2]: import h5py

In [3]: file = h5py.File('dragon_plumes.h5')

In [4]: fc_state = np.array(file.get('fc_state'))

In [5]: fc_state
Out[5]: 
array([['i', 'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i',
        'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i',
        'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i',
        'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i',
        'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i',
        'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i',
        'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i', 'a', 'a', 'a', 'a', 'a',
        'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a',
        'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a',
        'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a',
        'a', 'a']], 
      dtype='|S1')

Any thoughts?

Thanks!
Mike

--

"It is difficult to say what is impossible, for the dream of yesterday is the hope of today and the reality of tomorrow--Robert H. Goddard."

_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: writing std::strings to an hdf5 file

Binh-Minh Ribler

Hello Michael,


I just want to note that

    createDataSet() takes const DataType& data_type, not hid_t data_type

Please check other arguments as well.

You may also need H5Tset_size(data_type, H5T_VARIABLE).


Binh-Minh



From: Hdf-forum <[hidden email]> on behalf of Michael Barrucco <[hidden email]>
Sent: Monday, August 1, 2016 3:28 PM
To: [hidden email]
Subject: [Hdf-forum] writing std::strings to an hdf5 file
 
Hello Folks:

I am trying to write a string from a process in which the string changes at each iteration, and i dont know the size of the resulting string array at the time the write needs to happen.

I have some code, however, it only write the first character of the string on each iteration. Here is some sample code.


initialization============================================


std::string file_string = “foo.h5”;

H5File _file( file_name, H5F_ACC_TRUNC );

_col_chunk=0;


    DSetCreatPropList cparms_str;

    hsize_t chunk_dims_str[2] = {1,1};

    cparms_str.setChunk(2, chunk_dims_str);

    hid_t datatype = H5Tcopy(H5T_C_S1);

    char fill_val_str[1] = {""};

    cparms_str.setFillValue( datatype, &fill_val_str);

const H5std_string dataset_state(“string_data");

_dataset_state = _file.createDataSet(dataset_state,

                                         datatype, mspace_s, cparms_str);


Writing within the process===================================================


Get current string

hsize_t dimsext_s[2] = {1, 1};

const char * current_string_char = current_string.c_str();

size_s[0] = dimsext_s[0];

    size_s[1] = dimsext_s[1] + _col_chunk;

DataSpace filespace_state(_dataset_state.getSpace());

     filespace_state.selectHyperslab(H5S_SELECT_SET, dimsext_s, offset_s);

     DataSpace memspace_state(2, dimsext_s, NULL);

     hid_t datatype = H5Tcopy(H5T_C_S1);

    _dataset_state.write(currrent_string_char, datatype, memspace_state, filespace_state);


loading the resulting file==============================

In [1]: import numpy as np

In [2]: import h5py

In [3]: file = h5py.File('dragon_plumes.h5')

In [4]: fc_state = np.array(file.get('fc_state'))

In [5]: fc_state
Out[5]: 
array([['i', 'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i',
        'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i',
        'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i',
        'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i',
        'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i',
        'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i',
        'i', 'i', 'i', 'i', 'i', 'i', 'i', 'i', 'a', 'a', 'a', 'a', 'a',
        'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a',
        'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a',
        'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a',
        'a', 'a']], 
      dtype='|S1')

Any thoughts?

Thanks!
Mike

--

"It is difficult to say what is impossible, for the dream of yesterday is the hope of today and the reality of tomorrow--Robert H. Goddard."

_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
Loading...