How to read Fixed Length String Dataset values

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

How to read Fixed Length String Dataset values

Deepak 8 Kumar
I am reading hdf5 file which has String dataset values with fixed length size and the number of dimensions is 1. I am using HDF 1.10.1 Windows10 x64.
I allocate array of pointers to read the data values in the same way as given in the hdf5 documentation. Here is the code which is reading the string dataset values.
In this code, the dynamic 2D array of pointers is not initialized as per the standard way using a loop.

unique_ptr<T*[]> apbuffer = make_unique<T*[]>(size_of_dimensions[0]);
T** buffer = dpbuffer.get();
unique_ptr<T[]> apbuffer1 = make_unique<T[]>(size_of_dimensions[0] * size);
buffer[0] = dpbuffer1.get();
for (int i = 1; i < size_of_dimensions[0]; i++)  {
        buffer[i] = buffer[0] + i * size_of_dimensions[1];
}
H5Dread(dataset_id, dataset_type_id, H5S_ALL, H5S_ALL, H5P_DEFAULT, &buffer[0][0]);

I would like to know, what are other possible ways to allocate the buffer to read 1 dimension string dataset values?
Can we simply read the 1 dimension dataset values by passing arrays pointed to the buffer?
Any insight is greatly appreciated.

_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
Reply | Threaded
Open this post in threaded view
|

Re: How to read Fixed Length String Dataset values

Werner Benger

Hi,

 it's easier to have just one contiguous buffer of type char* with a single allocation for doing the I/O. If your strings are all of fixed length, you can pack all those independent strings into a single char* buffer . Then do I/O on that one-dimensional buffer of chars, and when you need them as strings, you extract them from this char* buffer.


On 27.07.2017 20:49, Deepak 8 Kumar wrote:
I am reading hdf5 file which has String dataset values with fixed length size and the number of dimensions is 1. I am using HDF 1.10.1 Windows10 x64.
I allocate array of pointers to read the data values in the same way as given in the hdf5 documentation. Here is the code which is reading the string dataset values.
In this code, the dynamic 2D array of pointers is not initialized as per the standard way using a loop.

unique_ptr<T*[]> apbuffer = make_unique<T*[]>(size_of_dimensions[0]);
T** buffer = dpbuffer.get();
unique_ptr<T[]> apbuffer1 = make_unique<T[]>(size_of_dimensions[0] * size);
buffer[0] = dpbuffer1.get();
for (int i = 1; i < size_of_dimensions[0]; i++)  {
        buffer[i] = buffer[0] + i * size_of_dimensions[1];
}
H5Dread(dataset_id, dataset_type_id, H5S_ALL, H5S_ALL, H5P_DEFAULT, &buffer[0][0]);

I would like to know, what are other possible ways to allocate the buffer to read 1 dimension string dataset values?
Can we simply read the 1 dimension dataset values by passing arrays pointed to the buffer?
Any insight is greatly appreciated.


_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5

-- 
___________________________________________________________________________
Dr. Werner Benger                Visualization Research
Center for Computation & Technology at Louisiana State University (CCT/LSU)
2019  Digital Media Center, Baton Rouge, Louisiana 70803
Tel.: +1 225 578 4809                        Fax.: +1 225 578-5362 

_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
Reply | Threaded
Open this post in threaded view
|

Re: How to read Fixed Length String Dataset values

Deepak 8 Kumar
If I allocate just one contiguous buffer of type char*, it reads only the first element. Rest of the elements are not in the buffer.
unique_ptr<char[]> pdataset_buffer = make_unique<char[]>(number_of_elements * size);
char* buffer = pdataset_buffer.get();
HDF5_Utility::check_error(H5Dread(dataset_id, dataset_type_id, H5S_ALL, H5S_ALL, H5P_DEFAULT, buffer));



From:        Werner Benger <[hidden email]>
To:        [hidden email]
Date:        07/28/2017 05:56 AM
Subject:        Re: [Hdf-forum] How to read Fixed Length String Dataset values
Sent by:        "Hdf-forum" <[hidden email]>




Hi,

 it's easier to have just one contiguous buffer of type char* with a single allocation for doing the I/O. If your strings are all of fixed length, you can pack all those independent strings into a single char* buffer . Then do I/O on that one-dimensional buffer of chars, and when you need them as strings, you extract them from this char* buffer.


On 27.07.2017 20:49, Deepak 8 Kumar wrote:
I am reading hdf5 file which has String dataset values with fixed length size and the number of dimensions is 1. I am using HDF 1.10.1 Windows10 x64.
I allocate array of pointers to read the data values in the same way as given in the hdf5 documentation. Here is the code which is reading the string dataset values.

In this code, the dynamic 2D array of pointers is not initialized as per the standard way using a loop.


unique_ptr<T*[]> apbuffer = make_unique<T*[]>(size_of_dimensions[0]);

T** buffer = dpbuffer.get();

unique_ptr<T[]> apbuffer1 = make_unique<T[]>(size_of_dimensions[0] * size);
buffer[0] = dpbuffer1.get();

for (int i = 1; i < size_of_dimensions[0]; i++)  {
        buffer[i] = buffer[0] + i * size_of_dimensions[1];

}

H5Dread(dataset_id, dataset_type_id, H5S_ALL, H5S_ALL, H5P_DEFAULT, &buffer[0][0]);


I would like to know, what are other possible ways to allocate the buffer to read 1 dimension string dataset values?

Can we simply read the 1 dimension dataset values by passing arrays pointed to the buffer?

Any insight is greatly appreciated.



_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter:
https://twitter.com/hdf5

--
___________________________________________________________________________
Dr. Werner Benger                Visualization Research
Center for Computation & Technology at Louisiana State University (CCT/LSU)
2019  Digital Media Center, Baton Rouge, Louisiana 70803
Tel.: +1 225 578 4809                        Fax.: +1 225 578-5362
_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.hdfgroup.org_mailman_listinfo_hdf-2Dforum-5Flists.hdfgroup.org&d=DQICAg&c=p0oa49nxxGtbbM2qgM-GB4r4m9OlGg-sEp8sXylY2aQ&r=aVpsmDSm2bBgznM4DES61bNo7E_uMhhiIutsE14aYRg&m=GbPA9wfVqnW0nhycDk86j7XOEAvoXpUlmquHwNHsaMA&s=uW7Cn-wF9HzlRqf8Aws8yJ6vb6MYG3KeyC5qXMKC8J8&e=
Twitter:
https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_hdf5&d=DQICAg&c=p0oa49nxxGtbbM2qgM-GB4r4m9OlGg-sEp8sXylY2aQ&r=aVpsmDSm2bBgznM4DES61bNo7E_uMhhiIutsE14aYRg&m=GbPA9wfVqnW0nhycDk86j7XOEAvoXpUlmquHwNHsaMA&s=m1WtgMPLWRuqo-Nvu7tCnphr1pRcY6cO86Qz48IW2zs&e=

_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5

Reply | Threaded
Open this post in threaded view
|

Re: How to read Fixed Length String Dataset values

Deepak 8 Kumar
In reply to this post by Werner Benger
I am getting all the elements in the buffer with  just one contiguous buffer of type char*. Thanks for all the help.



From:        Werner Benger <[hidden email]>
To:        [hidden email]
Date:        07/28/2017 05:56 AM
Subject:        Re: [Hdf-forum] How to read Fixed Length String Dataset values
Sent by:        "Hdf-forum" <[hidden email]>




Hi,

 it's easier to have just one contiguous buffer of type char* with a single allocation for doing the I/O. If your strings are all of fixed length, you can pack all those independent strings into a single char* buffer . Then do I/O on that one-dimensional buffer of chars, and when you need them as strings, you extract them from this char* buffer.


On 27.07.2017 20:49, Deepak 8 Kumar wrote:
I am reading hdf5 file which has String dataset values with fixed length size and the number of dimensions is 1. I am using HDF 1.10.1 Windows10 x64.
I allocate array of pointers to read the data values in the same way as given in the hdf5 documentation. Here is the code which is reading the string dataset values.

In this code, the dynamic 2D array of pointers is not initialized as per the standard way using a loop.


unique_ptr<T*[]> apbuffer = make_unique<T*[]>(size_of_dimensions[0]);

T** buffer = dpbuffer.get();

unique_ptr<T[]> apbuffer1 = make_unique<T[]>(size_of_dimensions[0] * size);
buffer[0] = dpbuffer1.get();

for (int i = 1; i < size_of_dimensions[0]; i++)  {
        buffer[i] = buffer[0] + i * size_of_dimensions[1];

}

H5Dread(dataset_id, dataset_type_id, H5S_ALL, H5S_ALL, H5P_DEFAULT, &buffer[0][0]);


I would like to know, what are other possible ways to allocate the buffer to read 1 dimension string dataset values?

Can we simply read the 1 dimension dataset values by passing arrays pointed to the buffer?

Any insight is greatly appreciated.



_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter:
https://twitter.com/hdf5

--
___________________________________________________________________________
Dr. Werner Benger                Visualization Research
Center for Computation & Technology at Louisiana State University (CCT/LSU)
2019  Digital Media Center, Baton Rouge, Louisiana 70803
Tel.: +1 225 578 4809                        Fax.: +1 225 578-5362
_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.hdfgroup.org_mailman_listinfo_hdf-2Dforum-5Flists.hdfgroup.org&d=DQICAg&c=p0oa49nxxGtbbM2qgM-GB4r4m9OlGg-sEp8sXylY2aQ&r=aVpsmDSm2bBgznM4DES61bNo7E_uMhhiIutsE14aYRg&m=GbPA9wfVqnW0nhycDk86j7XOEAvoXpUlmquHwNHsaMA&s=uW7Cn-wF9HzlRqf8Aws8yJ6vb6MYG3KeyC5qXMKC8J8&e=
Twitter:
https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_hdf5&d=DQICAg&c=p0oa49nxxGtbbM2qgM-GB4r4m9OlGg-sEp8sXylY2aQ&r=aVpsmDSm2bBgznM4DES61bNo7E_uMhhiIutsE14aYRg&m=GbPA9wfVqnW0nhycDk86j7XOEAvoXpUlmquHwNHsaMA&s=m1WtgMPLWRuqo-Nvu7tCnphr1pRcY6cO86Qz48IW2zs&e=

_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5