Quantcast

HDF5 API problems

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

HDF5 API problems

Rafal Lichwala
Hi,

For the purpose of some project I'd like to read HDF5 files in my C++
application using HDF5 API.

First of all I would like to read entire HDF5 file structure to know
where are the groups (their names, attributes etc.) and where I can find
datasets.
The API functions like H5*iterate (non-recursive version) or H5*visit
(recursive version) are rather useless, because their H5*_info_t
structures have very limited information about the object on the given
level in this structure.
Thus I've decided to walk manually through this structure and get as
much information as I can for the given object.

So...

Calling H5Gget_info gives me a number of links at the top ("." group)
which then I can ask for details based on their index. Fine.
Now, having the link index, I can call H5Lget_name_by_idx to get the
link name.
First initial call to that function will return the name size and second
one will return the link name in the properly prepared buffer of char*.

On that stage I found a first problem (a bug I suppose):
Let's assume the link name is "foobar", so it has 6 characters.
Initial call of H5Lget_name_by_idx returns 6, but when I call it again
with "size" parameter set to 6 it fills my buffer with just 5 characters
("fooba") even if the buffer has a proper size of 7 bytes (the 7th for
the final \0).
So it seems I have to increase size by 1 to get the proper link name.
What's the problem here? Maybe it's just my wrong understanding for this
API function?

Code example (returns "fooba" instead of "foobar" in the "link_name"
buffer):

// initial call to get name size
int link_name_size = H5Lget_name_by_idx(file.getId(), ".",
H5_INDEX_NAME, H5_ITER_NATIVE, i, NULL, 0, 0);

// one more byte for \0 at the end
char *link_name = new char[link_name_size + 1];

// final call
H5Lget_name_by_idx(file.getId(), ".", H5_INDEX_NAME, H5_ITER_NATIVE, i,
link_name, link_name_size, 0);


Can someone please explain me what's the problem here?

Regards,
Rafal


_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: HDF5 API problems

Binh-Minh Ribler

Hi Rafal,


You'll need to pass in link_name_size+1 to the final call of H5Lget_name_by_idx as well.


BTW, you can use h5dump to see the structure of the file, initially.


Thanks,

Binh-Minh



From: Hdf-forum <[hidden email]> on behalf of Rafal Lichwala <[hidden email]>
Sent: Tuesday, March 28, 2017 4:26 AM
To: [hidden email]
Subject: [Hdf-forum] HDF5 API problems
 
Hi,

For the purpose of some project I'd like to read HDF5 files in my C++
application using HDF5 API.

First of all I would like to read entire HDF5 file structure to know
where are the groups (their names, attributes etc.) and where I can find
datasets.
The API functions like H5*iterate (non-recursive version) or H5*visit
(recursive version) are rather useless, because their H5*_info_t
structures have very limited information about the object on the given
level in this structure.
Thus I've decided to walk manually through this structure and get as
much information as I can for the given object.

So...

Calling H5Gget_info gives me a number of links at the top ("." group)
which then I can ask for details based on their index. Fine.
Now, having the link index, I can call H5Lget_name_by_idx to get the
link name.
First initial call to that function will return the name size and second
one will return the link name in the properly prepared buffer of char*.

On that stage I found a first problem (a bug I suppose):
Let's assume the link name is "foobar", so it has 6 characters.
Initial call of H5Lget_name_by_idx returns 6, but when I call it again
with "size" parameter set to 6 it fills my buffer with just 5 characters
("fooba") even if the buffer has a proper size of 7 bytes (the 7th for
the final \0).
So it seems I have to increase size by 1 to get the proper link name.
What's the problem here? Maybe it's just my wrong understanding for this
API function?

Code example (returns "fooba" instead of "foobar" in the "link_name"
buffer):

// initial call to get name size
int link_name_size = H5Lget_name_by_idx(file.getId(), ".",
H5_INDEX_NAME, H5_ITER_NATIVE, i, NULL, 0, 0);

// one more byte for \0 at the end
char *link_name = new char[link_name_size + 1];

// final call
H5Lget_name_by_idx(file.getId(), ".", H5_INDEX_NAME, H5_ITER_NATIVE, i,
link_name, link_name_size, 0);


Can someone please explain me what's the problem here?

Regards,
Rafal


_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5

_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: HDF5 API problems

Rafal Lichwala
Hi Binh-Minh, All,

Thank you for your reply.

Yes, I know how to solve that finally, but I just think there is an
inconsistency in this (any many more) API functions.
First call to this function returns size without extra character for \0
at the end of the string, but second call to this function requires size
parameter to count this extra character for \0 at the end.
I think we should decide if we what to count this extra character for \0
or not and keep consistency in both function calls. So getting "size"
value from the first function call should be directly used in the second
call.
About the h5dump - sure - I know there are different tools which can
help and I actually use HDFView, but this is just for the time of
application development and testing, because the goal is to discover
HDF5 file structure in the application without using any external tools.

Anyway, It seems that was just a minor problem which I had with the HDF5
API, because now the next one looks worse...

Pure C-language API (function-based) seems to work more or less fine,
but when I try to use C++ API I can open the HDF5 file, but when I try
to get the object name (getObjName() method for the top group "." for
example) I get "read access violation" exception...

H5Object::getObjName() function internally uses pure C API where it
calls H5Iget_name() twice: first to get the name size (and build a
buffer for that name) and second to get the name itself. But then this
pure C char* buffer of characters is assigned to std::string which is
returned from that method. The problem is that maybe (but I'm not sure
here) due to some compiler optimizations (made while HDF libraries was
compiled - I just use binaries) this assignment of char* to std::string
is not made by copying the content, but simply reassigning the pointer.
Before this method returns, this char* array is deleted so that's why I
get "read access violation" exception when I try to get this string
content...

Can someone please help me in that problem?

This is the code from that method:

H5std_string H5Object::getObjName() const
{
     H5std_string obj_name(""); // object name to return

     // Preliminary call to get the size of the object name
     ssize_t name_size = H5Iget_name(getId(), NULL, static_cast<size_t>(0));

     // If H5Iget_name failed, throw exception
     if (name_size < 0)
     {
         throw Exception(inMemFunc("getObjName"), "H5Iget_name failed");
     }
     else if (name_size == 0)
     {
         throw Exception(inMemFunc("getObjName"), "Object must have a
name, but name length is 0");
     }
     // Object's name exists, retrieve it
     else if (name_size > 0)
     {
         char* name_C = new char[name_size+1];  // temporary C-string
         HDmemset(name_C, 0, name_size+1); // clear buffer

         // Use overloaded function
         name_size = getObjName(name_C, name_size+1);

         // Convert the C object name to return
         obj_name = name_C;

         // Clean up resource
         delete []name_C;
     }
     // Return object's name
     return(obj_name);
}

Regards,
Rafal



W dniu 2017-03-28 o 20:14, Binh-Minh Ribler pisze:

> Hi Rafal,
>
>
> You'll need to pass in link_name_size+1 to the final call
> of H5Lget_name_by_idx as well.
>
>
> BTW, you can use h5dump to see the structure of the file, initially.
>
>
> Thanks,
>
> Binh-Minh
>
>
> ------------------------------------------------------------------------
> *From:* Hdf-forum <[hidden email]> on behalf of
> Rafal Lichwala <[hidden email]>
> *Sent:* Tuesday, March 28, 2017 4:26 AM
> *To:* [hidden email]
> *Subject:* [Hdf-forum] HDF5 API problems
>
> Hi,
>
> For the purpose of some project I'd like to read HDF5 files in my C++
> application using HDF5 API.
>
> First of all I would like to read entire HDF5 file structure to know
> where are the groups (their names, attributes etc.) and where I can find
> datasets.
> The API functions like H5*iterate (non-recursive version) or H5*visit
> (recursive version) are rather useless, because their H5*_info_t
> structures have very limited information about the object on the given
> level in this structure.
> Thus I've decided to walk manually through this structure and get as
> much information as I can for the given object.
>
> So...
>
> Calling H5Gget_info gives me a number of links at the top ("." group)
> which then I can ask for details based on their index. Fine.
> Now, having the link index, I can call H5Lget_name_by_idx to get the
> link name.
> First initial call to that function will return the name size and second
> one will return the link name in the properly prepared buffer of char*.
>
> On that stage I found a first problem (a bug I suppose):
> Let's assume the link name is "foobar", so it has 6 characters.
> Initial call of H5Lget_name_by_idx returns 6, but when I call it again
> with "size" parameter set to 6 it fills my buffer with just 5 characters
> ("fooba") even if the buffer has a proper size of 7 bytes (the 7th for
> the final \0).
> So it seems I have to increase size by 1 to get the proper link name.
> What's the problem here? Maybe it's just my wrong understanding for this
> API function?
>
> Code example (returns "fooba" instead of "foobar" in the "link_name"
> buffer):
>
> // initial call to get name size
> int link_name_size = H5Lget_name_by_idx(file.getId(), ".",
> H5_INDEX_NAME, H5_ITER_NATIVE, i, NULL, 0, 0);
>
> // one more byte for \0 at the end
> char *link_name = new char[link_name_size + 1];
>
> // final call
> H5Lget_name_by_idx(file.getId(), ".", H5_INDEX_NAME, H5_ITER_NATIVE, i,
> link_name, link_name_size, 0);
>
>
> Can someone please explain me what's the problem here?
>
> Regards,
> Rafal
>
>
> _______________________________________________
> Hdf-forum is for HDF software users discussion.
> [hidden email]
> http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
> Twitter: https://twitter.com/hdf5
>
>
> _______________________________________________
> Hdf-forum is for HDF software users discussion.
> [hidden email]
> http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
> Twitter: https://twitter.com/hdf5
>

--

***
Rafał Lichwała
Poznańskie Centrum Superkomputerowo-Sieciowe
ul. Jana Pawła II nr 10
61-139 Poznań
e-mail: [hidden email]
***

_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: HDF5 API problems

Elvis Stansvik

Hi Rafal,

Den 29 mars 2017 8:21 fm skrev "Rafal Lichwala" <[hidden email]>:
>
> Hi Binh-Minh, All,
>
> Thank you for your reply.
>
> Yes, I know how to solve that finally, but I just think there is an inconsistency in this (any many more) API functions.
> First call to this function returns size without extra character for \0 at the end of the string, but second call to this function requires size parameter to count this extra character for \0 at the end.
> I think we should decide if we what to count this extra character for \0 or not and keep consistency in both function calls. So getting "size" value from the first function call should be directly used in the second call.

I think the API is just trying to be similar to strlen (and many other APIs), which always return the string length excluding the terminating null character.

If you think about it, it's logical; the terminating null character is not part of the string, it's simply, well, the terminator.

> About the h5dump - sure - I know there are different tools which can help and I actually use HDFView, but this is just for the time of application development and testing, because the goal is to discover HDF5 file structure in the application without using any external tools.
>
> Anyway, It seems that was just a minor problem which I had with the HDF5 API, because now the next one looks worse...
>
> Pure C-language API (function-based) seems to work more or less fine, but when I try to use C++ API I can open the HDF5 file, but when I try to get the object name (getObjName() method for the top group "." for example) I get "read access violation" exception...
>
> H5Object::getObjName() function internally uses pure C API where it calls H5Iget_name() twice: first to get the name size (and build a buffer for that name) and second to get the name itself. But then this pure C char* buffer of characters is assigned to std::string which is returned from that method. The problem is that maybe (but I'm not sure here) due to some compiler optimizations (made while HDF libraries was compiled - I just use binaries) this assignment of char* to std::string is not made by copying the content, but simply reassigning the pointer.

That would indicate a compiler bug, since operator= of std::string is defined to always make a copy. A compiler is not allowed to optimize that away if it results in a behavioral change from a user POV.

So I highly doubt that's the reason for your access violation.

It would help to see a minimal code example that exhibits the problem.

Cheers,
Elvis

> Before this method returns, this char* array is deleted so that's why I get "read access violation" exception when I try to get this string content...
>
> Can someone please help me in that problem?
>
> This is the code from that method:
>
> H5std_string H5Object::getObjName() const
> {
>     H5std_string obj_name(""); // object name to return
>
>     // Preliminary call to get the size of the object name
>     ssize_t name_size = H5Iget_name(getId(), NULL, static_cast<size_t>(0));
>
>     // If H5Iget_name failed, throw exception
>     if (name_size < 0)
>     {
>         throw Exception(inMemFunc("getObjName"), "H5Iget_name failed");
>     }
>     else if (name_size == 0)
>     {
>         throw Exception(inMemFunc("getObjName"), "Object must have a name, but name length is 0");
>     }
>     // Object's name exists, retrieve it
>     else if (name_size > 0)
>     {
>         char* name_C = new char[name_size+1];  // temporary C-string
>         HDmemset(name_C, 0, name_size+1); // clear buffer
>
>         // Use overloaded function
>         name_size = getObjName(name_C, name_size+1);
>
>         // Convert the C object name to return
>         obj_name = name_C;
>
>         // Clean up resource
>         delete []name_C;
>     }
>     // Return object's name
>     return(obj_name);
> }
>
> Regards,
> Rafal
>
>
>
> W dniu 2017-03-28 o 20:14, Binh-Minh Ribler pisze:
>>
>> Hi Rafal,
>>
>>
>> You'll need to pass in link_name_size+1 to the final call
>> of H5Lget_name_by_idx as well.
>>
>>
>> BTW, you can use h5dump to see the structure of the file, initially.
>>
>>
>> Thanks,
>>
>> Binh-Minh
>>
>>
>> ------------------------------------------------------------------------
>> *From:* Hdf-forum <[hidden email]> on behalf of
>> Rafal Lichwala <[hidden email]>
>> *Sent:* Tuesday, March 28, 2017 4:26 AM
>> *To:* [hidden email]
>> *Subject:* [Hdf-forum] HDF5 API problems
>>
>>
>> Hi,
>>
>> For the purpose of some project I'd like to read HDF5 files in my C++
>> application using HDF5 API.
>>
>> First of all I would like to read entire HDF5 file structure to know
>> where are the groups (their names, attributes etc.) and where I can find
>> datasets.
>> The API functions like H5*iterate (non-recursive version) or H5*visit
>> (recursive version) are rather useless, because their H5*_info_t
>> structures have very limited information about the object on the given
>> level in this structure.
>> Thus I've decided to walk manually through this structure and get as
>> much information as I can for the given object.
>>
>> So...
>>
>> Calling H5Gget_info gives me a number of links at the top ("." group)
>> which then I can ask for details based on their index. Fine.
>> Now, having the link index, I can call H5Lget_name_by_idx to get the
>> link name.
>> First initial call to that function will return the name size and second
>> one will return the link name in the properly prepared buffer of char*.
>>
>> On that stage I found a first problem (a bug I suppose):
>> Let's assume the link name is "foobar", so it has 6 characters.
>> Initial call of H5Lget_name_by_idx returns 6, but when I call it again
>> with "size" parameter set to 6 it fills my buffer with just 5 characters
>> ("fooba") even if the buffer has a proper size of 7 bytes (the 7th for
>> the final \0).
>> So it seems I have to increase size by 1 to get the proper link name.
>> What's the problem here? Maybe it's just my wrong understanding for this
>> API function?
>>
>> Code example (returns "fooba" instead of "foobar" in the "link_name"
>> buffer):
>>
>> // initial call to get name size
>> int link_name_size = H5Lget_name_by_idx(file.getId(), ".",
>> H5_INDEX_NAME, H5_ITER_NATIVE, i, NULL, 0, 0);
>>
>> // one more byte for \0 at the end
>> char *link_name = new char[link_name_size + 1];
>>
>> // final call
>> H5Lget_name_by_idx(file.getId(), ".", H5_INDEX_NAME, H5_ITER_NATIVE, i,
>> link_name, link_name_size, 0);
>>
>>
>> Can someone please explain me what's the problem here?
>>
>> Regards,
>> Rafal
>>
>>
>> _______________________________________________
>> Hdf-forum is for HDF software users discussion.
>> [hidden email]
>> http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
>> Twitter: https://twitter.com/hdf5
>>
>>
>> _______________________________________________
>> Hdf-forum is for HDF software users discussion.
>> [hidden email]
>> http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
>> Twitter: https://twitter.com/hdf5
>>
>
> --
>
> ***
> Rafał Lichwała
> Poznańskie Centrum Superkomputerowo-Sieciowe
> ul. Jana Pawła II nr 10
> 61-139 Poznań
> e-mail: [hidden email]
> ***
>
>
> _______________________________________________
> Hdf-forum is for HDF software users discussion.
> [hidden email]
> http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
> Twitter: https://twitter.com/hdf5


_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: HDF5 API problems

Rafal Lichwala
Hi Elvis,

Thank you for your reply!

W dniu 2017-03-29 o 08:41, Elvis Stansvik pisze:
> I think the API is just trying to be similar to strlen (and many other
> APIs), which always return the string length excluding the terminating
> null character.
>
> If you think about it, it's logical; the terminating null character is
> not part of the string, it's simply, well, the terminator.

I agree, but that means the second call to the H5Lget_name_by_idx (any
some others in API) does not behave as we want :-)
Let me explain on the example which I've sent previously: let's assume
the group name is "foobar".
So first call to H5Lget_name_by_idx() return 6 - that's correct - there
are 6 characters excluding NULL terminator.
So now I call H5Lget_name_by_idx again and pass 6 as a value of "size"
of expected name of this group. This time I just found "fooba" in the
buffer...
So I have to manually add 1 to size to get the whole name - that's not
intuitive I guess...

The documentation for H5Lget_name_by_idx function says:

"The size in bytes of name is specified in size. If size is unknown, it
can be determined via an initial H5Lget_name_by_idx call with name set
to NULL; the function's return value will be the size of the name."

To fix that simply, for that function (and some others in API) we could
just say: this function returns the number of characters in the name,
but expects the size of buffer (so +1 for NULL terminator).

Nevermind - that's really a minor issue - let's go to the next more
important one below...

> That would indicate a compiler bug, since operator= of std::string is
> defined to always make a copy. A compiler is not allowed to optimize
> that away if it results in a behavioral change from a user POV.

Well... I've made some research about that and it seems it's not so
simple. It seems that depending on the platform and compiler parameters,
sometimes it's possible to have different implementation od this
operator= for std::string.
I know - that sounds strange, but maybe that's the case this time.

> So I highly doubt that's the reason for your access violation.

I've just confirmed that there is something wrong with HDF5 binaries for
Windows x86 (ver. 1.10.0) (at least this version - maybe others too).
I've just downloaded HDF5 sources and compiled them locally (Visual
Studio 2017 with CMake support). Then I use those DLL's in my project
and... it works! No access violation and the group name is properly
returned.

>
> It would help to see a minimal code example that exhibits the problem.

Sure. Here it is:

#include <iostream>
#include "H5Cpp.h"

using namespace std;

#ifndef H5_NO_NAMESPACE
        using namespace H5;
#endif
#define FILE_NAME "new.h5"

int main()
{
        try {
                if (H5File::isHdf5(FILE_NAME))
                {
                        H5File file(FILE_NAME, H5F_ACC_RDONLY);
                        Group g = file.openGroup(".");

                        cout << "Nr of links: " << g.getNumObjs() << endl;

                        for (hsize_t i = 0; i < g.getNumObjs(); i++)
                        {
                                string name = g.getObjnameByIdx(i);

                                // that's the place where access violation comes with DLL binaries used
                                cout << "Link name: " << name << endl;
                        }

                        g.close();
                        file.close();
                }
        }
        catch (FileIException error)
        {
                error.printError();
                return -1;
        }
        catch (DataSetIException error)
        {
                error.printError();
                return -1;
        }
        return 0;
}


Regards,
Rafal


_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: HDF5 API problems

Rafal Lichwala
Hi All,

I think I found a nice explanation of my problems with Windows binaries
of HDF5 library described before:

http://stackoverflow.com/questions/5339811/how-can-i-pass-c-objects-to-dlls-with-different-iterator-debug-level/5340065#5340065

So it seems I have to use recompiled version of that C++ API or just
forget about C++ and use pure C API instead.

BTW here is a nice article about how to properly implement C++ library
and be able to use them in Windows DLL world:

https://www.codeproject.com/Articles/28969/HowTo-Export-C-classes-from-a-DLL#CppMatureApproach

How it looks in the Unix/Linux world?
Are there similar problems in using binaries of those libraries?
I have to check that by my own anyway... :-)

Regards,
Rafal


W dniu 2017-03-29 o 09:39, Rafal Lichwala pisze:

> Hi Elvis,
>
> Thank you for your reply!
>
> W dniu 2017-03-29 o 08:41, Elvis Stansvik pisze:
>> I think the API is just trying to be similar to strlen (and many other
>> APIs), which always return the string length excluding the terminating
>> null character.
>>
>> If you think about it, it's logical; the terminating null character is
>> not part of the string, it's simply, well, the terminator.
>
> I agree, but that means the second call to the H5Lget_name_by_idx (any
> some others in API) does not behave as we want :-)
> Let me explain on the example which I've sent previously: let's assume
> the group name is "foobar".
> So first call to H5Lget_name_by_idx() return 6 - that's correct - there
> are 6 characters excluding NULL terminator.
> So now I call H5Lget_name_by_idx again and pass 6 as a value of "size"
> of expected name of this group. This time I just found "fooba" in the
> buffer...
> So I have to manually add 1 to size to get the whole name - that's not
> intuitive I guess...
>
> The documentation for H5Lget_name_by_idx function says:
>
> "The size in bytes of name is specified in size. If size is unknown, it
> can be determined via an initial H5Lget_name_by_idx call with name set
> to NULL; the function's return value will be the size of the name."
>
> To fix that simply, for that function (and some others in API) we could
> just say: this function returns the number of characters in the name,
> but expects the size of buffer (so +1 for NULL terminator).
>
> Nevermind - that's really a minor issue - let's go to the next more
> important one below...
>
>> That would indicate a compiler bug, since operator= of std::string is
>> defined to always make a copy. A compiler is not allowed to optimize
>> that away if it results in a behavioral change from a user POV.
>
> Well... I've made some research about that and it seems it's not so
> simple. It seems that depending on the platform and compiler parameters,
> sometimes it's possible to have different implementation od this
> operator= for std::string.
> I know - that sounds strange, but maybe that's the case this time.
>
>> So I highly doubt that's the reason for your access violation.
>
> I've just confirmed that there is something wrong with HDF5 binaries for
> Windows x86 (ver. 1.10.0) (at least this version - maybe others too).
> I've just downloaded HDF5 sources and compiled them locally (Visual
> Studio 2017 with CMake support). Then I use those DLL's in my project
> and... it works! No access violation and the group name is properly
> returned.
>
>>
>> It would help to see a minimal code example that exhibits the problem.
>
> Sure. Here it is:
>
> #include <iostream>
> #include "H5Cpp.h"
>
> using namespace std;
>
> #ifndef H5_NO_NAMESPACE
>     using namespace H5;
> #endif
> #define FILE_NAME "new.h5"
>
> int main()
> {
>     try {
>         if (H5File::isHdf5(FILE_NAME))
>         {
>             H5File file(FILE_NAME, H5F_ACC_RDONLY);
>             Group g = file.openGroup(".");
>
>             cout << "Nr of links: " << g.getNumObjs() << endl;
>
>             for (hsize_t i = 0; i < g.getNumObjs(); i++)
>             {
>                 string name = g.getObjnameByIdx(i);
>
>                 // that's the place where access violation comes with
> DLL binaries used
>                 cout << "Link name: " << name << endl;
>             }
>
>             g.close();
>             file.close();
>         }
>     }
>     catch (FileIException error)
>     {
>         error.printError();
>         return -1;
>     }
>     catch (DataSetIException error)
>     {
>         error.printError();
>         return -1;
>     }
>     return 0;
> }
>
>
> Regards,
> Rafal
>
>
> _______________________________________________
> Hdf-forum is for HDF software users discussion.
> [hidden email]
> http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
> Twitter: https://twitter.com/hdf5

--

***
Rafał Lichwała
Poznańskie Centrum Superkomputerowo-Sieciowe
ul. Jana Pawła II nr 10
61-139 Poznań
e-mail: [hidden email]
***

_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: HDF5 API problems

Rafal Lichwala
Regarding Unix/Linux environment, it seems HDF5 library does not like my
NFS:

"unable to flock file, errno = 37, error message = 'No locks available'"

Is there any way to force this library to use lockf() instead of flock()
which is not available on NFS?
If any, I guess I have to recompile the library anyway - right?

Regards,
Rafal



W dniu 2017-03-29 o 11:44, Rafal Lichwala pisze:

> Hi All,
>
> I think I found a nice explanation of my problems with Windows binaries
> of HDF5 library described before:
>
> http://stackoverflow.com/questions/5339811/how-can-i-pass-c-objects-to-dlls-with-different-iterator-debug-level/5340065#5340065
>
>
> So it seems I have to use recompiled version of that C++ API or just
> forget about C++ and use pure C API instead.
>
> BTW here is a nice article about how to properly implement C++ library
> and be able to use them in Windows DLL world:
>
> https://www.codeproject.com/Articles/28969/HowTo-Export-C-classes-from-a-DLL#CppMatureApproach
>
>
> How it looks in the Unix/Linux world?
> Are there similar problems in using binaries of those libraries?
> I have to check that by my own anyway... :-)
>
> Regards,
> Rafal
>
>
> W dniu 2017-03-29 o 09:39, Rafal Lichwala pisze:
>> Hi Elvis,
>>
>> Thank you for your reply!
>>
>> W dniu 2017-03-29 o 08:41, Elvis Stansvik pisze:
>>> I think the API is just trying to be similar to strlen (and many other
>>> APIs), which always return the string length excluding the terminating
>>> null character.
>>>
>>> If you think about it, it's logical; the terminating null character is
>>> not part of the string, it's simply, well, the terminator.
>>
>> I agree, but that means the second call to the H5Lget_name_by_idx (any
>> some others in API) does not behave as we want :-)
>> Let me explain on the example which I've sent previously: let's assume
>> the group name is "foobar".
>> So first call to H5Lget_name_by_idx() return 6 - that's correct - there
>> are 6 characters excluding NULL terminator.
>> So now I call H5Lget_name_by_idx again and pass 6 as a value of "size"
>> of expected name of this group. This time I just found "fooba" in the
>> buffer...
>> So I have to manually add 1 to size to get the whole name - that's not
>> intuitive I guess...
>>
>> The documentation for H5Lget_name_by_idx function says:
>>
>> "The size in bytes of name is specified in size. If size is unknown, it
>> can be determined via an initial H5Lget_name_by_idx call with name set
>> to NULL; the function's return value will be the size of the name."
>>
>> To fix that simply, for that function (and some others in API) we could
>> just say: this function returns the number of characters in the name,
>> but expects the size of buffer (so +1 for NULL terminator).
>>
>> Nevermind - that's really a minor issue - let's go to the next more
>> important one below...
>>
>>> That would indicate a compiler bug, since operator= of std::string is
>>> defined to always make a copy. A compiler is not allowed to optimize
>>> that away if it results in a behavioral change from a user POV.
>>
>> Well... I've made some research about that and it seems it's not so
>> simple. It seems that depending on the platform and compiler parameters,
>> sometimes it's possible to have different implementation od this
>> operator= for std::string.
>> I know - that sounds strange, but maybe that's the case this time.
>>
>>> So I highly doubt that's the reason for your access violation.
>>
>> I've just confirmed that there is something wrong with HDF5 binaries for
>> Windows x86 (ver. 1.10.0) (at least this version - maybe others too).
>> I've just downloaded HDF5 sources and compiled them locally (Visual
>> Studio 2017 with CMake support). Then I use those DLL's in my project
>> and... it works! No access violation and the group name is properly
>> returned.
>>
>>>
>>> It would help to see a minimal code example that exhibits the problem.
>>
>> Sure. Here it is:
>>
>> #include <iostream>
>> #include "H5Cpp.h"
>>
>> using namespace std;
>>
>> #ifndef H5_NO_NAMESPACE
>>     using namespace H5;
>> #endif
>> #define FILE_NAME "new.h5"
>>
>> int main()
>> {
>>     try {
>>         if (H5File::isHdf5(FILE_NAME))
>>         {
>>             H5File file(FILE_NAME, H5F_ACC_RDONLY);
>>             Group g = file.openGroup(".");
>>
>>             cout << "Nr of links: " << g.getNumObjs() << endl;
>>
>>             for (hsize_t i = 0; i < g.getNumObjs(); i++)
>>             {
>>                 string name = g.getObjnameByIdx(i);
>>
>>                 // that's the place where access violation comes with
>> DLL binaries used
>>                 cout << "Link name: " << name << endl;
>>             }
>>
>>             g.close();
>>             file.close();
>>         }
>>     }
>>     catch (FileIException error)
>>     {
>>         error.printError();
>>         return -1;
>>     }
>>     catch (DataSetIException error)
>>     {
>>         error.printError();
>>         return -1;
>>     }
>>     return 0;
>> }
>>
>>
>> Regards,
>> Rafal
>>
>>
>> _______________________________________________
>> Hdf-forum is for HDF software users discussion.
>> [hidden email]
>> http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
>> Twitter: https://twitter.com/hdf5
>

--

***
Rafał Lichwała
Poznańskie Centrum Superkomputerowo-Sieciowe
ul. Jana Pawła II nr 10
61-139 Poznań
e-mail: [hidden email]
***

_______________________________________________
Hdf-forum is for HDF software users discussion.
[hidden email]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
Loading...